Python websocket doesn't get the right live data

[2803media]

Hi
I use the websocket with a python 3.6 script and I notice that after some couple of hours of running the data is not the real live value for example right now the live BTC is 7897 and I get 7922 with websocket (the script was launched 12 hours ago).

Here is my script:

from bitmex_websocket import BitMEXWebsocket
import logging
from logging.handlers import RotatingFileHandler
from time import sleep
import datetime
import json

crypto = "XBTUSD"
api_key = "XXXXX"
api_secret = "YYYYY"
sleeping = 5

def run():
    logger = setup_logger()

    ws = BitMEXWebsocket(endpoint="https://www.bitmex.com/api/v1",
                         symbol=crypto, api_key=api_key, api_secret=api_secret)
    logger.info("Instrument data: %s" % ws.get_instrument())
    while(ws.ws.sock.connected):
        logger.info("Ticker: %s" % ws.get_ticker())
        sleep(sleeping)

def setup_logger():
    # Prints logger info to terminal
    logger = logging.getLogger()
    # Change this to DEBUG if you want a lot more info
    logger.setLevel(logging.INFO)
    ch = logging.StreamHandler()
    # create formatter
    formatter = logging.Formatter(
        "%(asctime)s - %(name)s - %(levelname)s - %(message)s")
    file_handler = RotatingFileHandler(
        '/home/test/public_html/block/bot/_activity_' + api_key + '.log', 'a', 1000000, 1)
    file_handler.setLevel(logging.DEBUG)
    file_handler.setFormatter(formatter)
    logger.addHandler(file_handler)
    # add formatter to ch
    ch.setFormatter(formatter)
    logger.addHandler(ch)
    return logger

if __name__ == "__main__":
    run()

Thanks for your feedback on this lag or bug !

[ryanfox]

Some more information would be helpful to diagnose what's going on. Do you see any error messages? Is the websocket still connected when you see the discrepancy? Does the delta grow over time?

What data point (bid, ask, last price, etc) are you seeing the difference in?

[2803media]

Yep the websocket is still connect, I don't see any error messages and the delta doesn't grow, it's just not match the live data. And I watch the 'last price'.

I notice that my server CPU is heavily used (not related of the websocket but with other operations), I just try to lower the usage to see if I get delta. Hope it help you?

[2803media]

So a couple of hours later the delta is now near 0, so when the CPU is overloaded the websocket get some delta with the live data...

I don't know if it's logical and I don't know why... But the problem is solved

[2803media]

Bad new it wasn't that... I still have delta.... So forget the CPU thing it wasn't the problem...

Websocket last price: 7496
Live price: 7511.0

[ryanfox]

That script doesn't print the new value immediately when messages are received. You're effectively polling every 5 seconds - it's certainly possible for the last price to move 15 points in that amount of time.

[2803media]

Thanks I will try with out the sleep of 5 seconds! And I will post an update here.

[scoshil]

@ryanfox I have the same issue with the python websocket. The orderbook data appears to come through fine, but when there is sudden movement in the market the data keeps coming through without an error, however it is obvious that the data delayed, by 60+ seconds (compared to the bitmex website on the same computer). For example, website price goes 7500>7530>7510, then 60 seconds later I see the same data come through on websocket.

Is there a way to find out what the server time is when messages are sent out? Or more generally a way to detect whether this is delayed data or whether there is a problem on my end?

Thanks

[ryanfox]

Many of the websocket messages have a timestamp field - that is the time the server created the message. You could compare that timestamp to the time you receive the message.

[scoshil]

@ryanfox so I compared server timestamps for orderbook10 and position messages (I was previously using orderBookL2 updates but they don't contain timestamps) and it seems like whenever the exchange gets busy, the time lag between server time and when I get the message drifts out from the baseline value by >10 seconds, and later when the exchange settles down, the backlog is slowly cleared.

What I'm confused by is where this backlog comes from?

I don't think it's a problem with my internet connection because I have the website up simultaneously and the website does not appear delayed at all. This would also suggest it's not a server side issue, unless the website gets some special treatment over websocket apis?
Also, the on_message callback for messages takes <1ms to process each message on average, so I don't think that would be the problem.

Is there something else within the library that could be causing this delay?

EDIT: I should also add that I'm counting roughly 30-60 websocket msgs/sec on my connection, so with an average processing time of <1ms I am very confused as to why I'm seeing 10 secs delay on websocket vs website when exchange gets busy.

[ryanfox]

You're saying in __on_message the timestamps are already 10 seconds old by the time you see them?

[scoshil]

Yes. On further investigation, I have found that running the bitmex websocket by itself I do not encounter this issue. I see peak msgs/sec of around 230 and there is never a delay more than 1 second.

The issue seems to stem from the fact that I run the bitmex websocket concurrently with other websockets. It seems that having 2 different websockets running reduces the speed of the bitmex websocket to handle messages, and thus they get queued up somehow.....but seeing as the ws.run_forever() is on a thread I don't understand what is causing this limitation.

Any ideas would be appreciated!

[2803media]

I run multi instances too!
My solution is to reload script each 12h, it's not the best method but fill my needs.

[ryanfox]

The web interface and python client are using the same API, but are separate connections - their requests may be served at different times.

When the system is under load there may be latency between the trading engine and clients. It's a known performance issue - it's being worked on along with general performance improvements.

[scoshil]

To clarify my understanding of what you just said about same API/diff connections.

Are you saying it's a known issue that the web interface could be faster than the websocket by >10 seconds when Bitmex is under heavy load?

If there are 5000 websocket connections via API and 1000 via website, would you expect the website to be faster?

[ryanfox]

Are you saying it's a known issue that the web interface could be faster than the websocket by >10 seconds when Bitmex is under heavy load?

The issue is that latency can accumulate between the engine and clients under server load.

If there are 5000 websocket connections via API and 1000 via website, would you expect the website to be faster?

The website might have less lag, or the python client might. They both use the same websocket API - from the server's perspective, they are both clients to serve.

[asiva]

I am seeing the same problem, also on python using BitMEXWebsocket. I have not seen the website lag behind the python client, always the other way around. It is currently happening very often and very quickly --- within a few minutes of a restart with lags of a few minutes. A restart of the websocket clears the lag immediately.

[Steve-1975]

I am having the same issue. Delays of up to 10 seconds and restarting clears them.
After running for a couple of hours, the orderbook is scrambled to the point where nothing matches up. Is there a clean way to restart this ws-socket without relaunching the whole thing.

[scoshil]

to echo above comments:

1. I am also using Bitmex websocket
2. I also never see the website lag the python websocket client
3. I have seen delays >30 seconds, The delay usually gets bigger the busier the market is.
4. After things quieten down, the delay eventually (could be 10+ minutes later) returns back down to normal (zero-ish).

It seems like something somewhere is catching up over time, but I can't tell what it is (from my end).

[Steve-1975]

Just notice that the orderbook volumes for the first tick up/down are completely off at all times.

[asiva]

I'm no expert, YMMV:

The bitmex_websocket implementation relies on the websockets library. It is possible that this is a performance issue. See for example this discussion about lagging websocket collection from bitfinex.

I think it's the computation speed, not the network connection because running the DeltaServer/nodejs implementation on the same computer instead faces no lagging issues.

If you're sticking with python and don't need orderbookl2, remove it. It's by far the slowest/largest set of data, although you will have to remove/modify all references to orderbookl2 in the code to get it to run. Also, get_ticker uses the quote feed to get the best bid/ask and volumes but rounds the prices to the nearest dollar(!). Remove the rounding.

[Steve-1975]

I observe the same thing here. The NodeJS behavior is ok.

[scoshil]

@asiva thanks that is very useful to know. Have you tested whether the problem goes away/is less noticeable if you:

1. run without bitfinex
2. Remove orderBookL2 and use only quote
3. run with more compute resources

Regarding the delta server, (sorry if this is a noob question but I only know python) is it possible to run it and get the python script to fetch the latest data on demand locally?

[Steve-1975]

@scoshil Regarding the delta-server. Yes, works with python. Just follow the install procedure from the readme. It's very easy. It exposes a socket that returns JSON on your localhost. Check the curl examples.
curl "http://localhost:4444/instrument?symbol=XBTUSD"

[scoshil]

I'm trying to get the Node-js delta-server running.

When I run: node index.js I get the following output:

BitMEX-Delta-Server listening on http://localhost:4444
WebSocket [INFO]: Connected.
{"op":"subscribe","args":"instrument:XBTUSD"}
{"op":"subscribe","args":"orderBookL2:XBTUSD"}
{"op":"subscribe","args":"quote:XBTUSD"}
{"op":"subscribe","args":"trade:XBTUSD"}

The command I'm using in python is:

import requests
response = requests.get('http://localhost:4444/orderBookL2')

print(response)

which returns a 500msg and the below output in the server:

TypeError: data.map is not a function
    at clone (C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\nodejs\index.js:231:15)
    at BitMEXClient.getData (C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\nodejs\index.js:98:10)
    at C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\delta-server\lib\server.js:76:21
    at Layer.handle [as handle_request] (C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\layer.js:95:5)
    at next (C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\route.js:137:13)
    at Route.dispatch (C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\route.js:112:3)
    at Layer.handle [as handle_request] (C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\layer.js:95:5)
    at C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:281:22
    at param (C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:354:14)
    at param (C:\Users\2lesh\Downloads\bitmex\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:365:14)

Sorry if this is a beginner question but I don't really understand what this error means/how to fix it

Additional info:
When I was doing the npm install commands it kept saying there was a high vulnerability found, and I needed to do npm audit --fix, so I did that...would that have broken this? If so, how do I fix it?

[asiva]

@scoshil FYI delta-server struggles with orderbookL2 data on my computer. I haven't had the spare time to experiment with more/less resources etc.

What version of nodejs did you install? I believe it has to be a version 6. I have node 6.14.2 and do not have the audit package installed by default. Perhaps try reinstalling?

[scoshil]

ah....I just went to nodejs.org and downloaded the latest version...it appears it's v8.11.3.....is that a similiar thing to python 2 vs python 3 where there is not necessarily backwards compatibility?

EDIT: Just tried with Node v6.14.2 and getting same error...

[TomDwz]

I have the same problem with BitMEXClient.getData. It gives me the same error as data.map is not a function. Here is the code:
client.addStream('XBTUSD', 'trade', () => {}); setTimeout(() => { console.log('XBTUSD trades during the last few seconds:', client.getTable('trade').XBTUSD); }, 5000);

[TomDwz]

I just changed the index.js of this module and now it works. I am not sure if this change will affect other components in this module. It at least fixes the error now.

//replace clone function in index.js with this function clone(data) { if (!Array.isArray(data)) { data = Object.values(data)[0] } return data.map(o => Object.assign({}, o)); }

[scoshil]

thanks @TomDwz ...I just tried your suggestion and now when I run node index.js the output is:

TypeError: Object.values is not a function
    at clone (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\nodejs\index.js:234:66)
    at BitMEXClient.getData (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\nodejs\index.js:98:10)
    at C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\lib\server.js:76:21    at Layer.handle [as handle_request] (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\layer.js:95:5)
    at next (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\route.js:137:13)
    at Route.dispatch (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\route.js:112:3)
    at Layer.handle [as handle_request] (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\layer.js:95:5)
    at C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:281:22
    at param (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:354:14)
    at param (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:365:14)
TypeError: Object.values is not a function
    at clone (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\nodejs\index.js:234:66)
    at BitMEXClient.getData (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\nodejs\index.js:98:10)
    at C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\lib\server.js:76:21    at Layer.handle [as handle_request] (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\layer.js:95:5)
    at next (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\route.js:137:13)
    at Route.dispatch (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\route.js:112:3)
    at Layer.handle [as handle_request] (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\layer.js:95:5)
    at C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:281:22
    at param (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:354:14)
    at param (C:\Users\2lesh\Downloads\api-connectors-master\api-connectors-master\official-ws\delta-server\node_modules\express\lib\router\index.js:365:14)

Also, this might be a stupid question but I am supposed to run >>node index.js in \official-ws\delta-server right? I ask because if I run >>node example.js in \official-ws\nodejs I get data flowing (but then the problem is I can't fetch that data when I run a request).

EDIT: I decided to just make BitMEXClient.getData return 'out' instead of 'clone(out)' and now it seems to work. Am I correct in saying that if I fetch this data using python I won't run into any modification issues? Or should I still use .copy() in python?

[TomDwz]

I think the best way is to check the clone function in your index.js again. You can do a "console.log(typeof data)" before the return statement and see if data is an object type or not. The reason I add the if statement to the function is because in my program the data parameter is an object and object doesn't have a function for .map(). And that causes the error. So I need to cast it to an array in order to use the .map() function. Hope you can find your error there too.

[cemdev]

blu - one thing to check is your code itself. if it's synchronous then the execution time of whatever you're doing with the data will cause the 'dropped' comms. you may well only be seeing it on xbtusd because the volume is so much greater, so more trades comes through. as soon as a message is received, pawn it off to another process and you'll see your throughput increase.

[letusfly85]

Hi, I'm also annoyed by delaying python web socket library.

Now, I'm trying to use Node.js library, and it doesn't seem to delay against web view.

• https://www.bitmex.com/app/trade/XBTUSD

By python scripts, in first 2 or 3 minutes o.k. however, after that delay or lags become larger.
Is there any solutions...? 👀

[tsoonami]

Hi, I am having a similar-ish issue. I've been able to get the delta server setup, but for some reason am getting an error in the execution table. All other tables (including the authenticated ones) work fine.

Has anyone run into this problem? The error readout I get when I try clicking on the execution table via the web interface is below:

TypeError: Cannot read property 'XBTUSD' of undefined
    at BitMEXClient.getData (C:\Users\Administrator\api-connectors\official-ws\nodejs\index.js:84:32)
    at C:\Users\Administrator\api-connectors\official-ws\delta-server\lib\server.js:76:21
    at Layer.handle [as handle_request] (C:\Users\Administrator\api-connectors\official-ws\delta-server\node_modules\express\lib\router\layer.js:95:5)
    at next (C:\Users\Administrator\api-connectors\official-ws\delta-server\node_modules\express\lib\router\route.js:137:13)
    at Route.dispatch (C:\Users\Administrator\api-connectors\official-ws\delta-server\node_modules\express\lib\router\route.js:112:3)
    at Layer.handle [as handle_request] (C:\Users\Administrator\api-connectors\official-ws\delta-server\node_modules\express\lib\router\layer.js:95:5)
    at C:\Users\Administrator\api-connectors\official-ws\delta-server\node_modules\express\lib\router\index.js:281:22
    at param (C:\Users\Administrator\api-connectors\official-ws\delta-server\node_modules\express\lib\router\index.js:354:14)
    at param (C:\Users\Administrator\api-connectors\official-ws\delta-server\node_modules\express\lib\router\index.js:365:14)
    at Function.process_params (C:\Users\Administrator\api-connectors\official-ws\delta-server\node_modules\express\lib\router\index.js:410:3) ```

[bluworld]

scoshil,

this seemed to work for me with delta-server:

1. install delta-server per instructions: https://github.com/BitMEX/api-connectors/tree/master/official-ws/delta-server

2. upgrade nodejs to v10: https://tecadmin.net/install-latest-nodejs-npm-on-ubuntu/

3. To listen to the node publisher in python:

import requests
response = requests.get('http://localhost:4444/instrument?symbol=XBTUSD')
data = response.json()
print(data)

Replace /instrument with /orderBookL2 etc

The market feed is now much better.

[paltsev-p]

@bluworld ,
Could you advice what result do you get if you subscribe to a private channel ("order" or "execution") and try to fetch this data at http://localhost:4444/order or "http://localhost:4444/execution ?

PS: Depending on the index.js content I'm receiving two different errors:

1. If I keep the aforementioned index.js as it's downloaded from GitHub:
   TypeError: data.map is not a function at clone (/api-connectors/official-ws/nodejs/index.js:231:15) ....

2. Another error appears if I make the changes recommended by @TomDwz here

TypeError: Cannot read property 'map' of undefined at clone (/api-connectors/official-ws/nodejs/index.js:231:15) ....

[bluworld]

@paltsev-p I have not set up private subscriptions yet, but I am getting this error when my python client uses requests to listen to the delta server. I get this error when using urllib or request module. It seems to occur from time to time. Seems like its an issue with the publishing server? Looking into this now but if anyone else has any ideas, that would be great.

Anyone on the bitmex team here that can help?

Traceback when using urllib:

Traceback (most recent call last):
File "prod.py", line 34, in
instrument = urllib.request.urlopen('http://localhost:4444/instrument?symbol=XBTU18')
File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 1282, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.5/urllib/request.py", line 1257, in do_open
r = h.getresponse()
File "/usr/lib/python3.5/http/client.py", line 1197, in getresponse
response.begin()
File "/usr/lib/python3.5/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.5/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

Traceback when using Request:

Traceback (most recent call last):
File "prod.py", line 33, in
orderbook = requests.get('http://localhost:4444/orderBookL2?symbol=XBTUSD')
File "/usr/local/lib/python3.5/dist-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/requests/adapters.py", line 495, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

[vesslanjin]

have the same issue, I received absolute wrong data after long run (like after several hours), it appears ask1 price is less than bid1 price. I used pure websocket API, I am not using the delta server.

[alexmrs94]

@tsoonami have you found a solution? I have setup the delta server, but I get the same error as you whenever I'm trying to get authenticated tables...

[ronanmendes]

hi. I am 100% layman in coding. I contract a dev to wirte a bot for bitmex. I have several problems with quotes (my bot make several calls by minute and sometimes per second).

the quotes aways have a delay of at least 2 seconds

and i NEVER get xxxx.5 prices, only xxxx

anyone got this problem with .5 prices???

[anedosugov]

after some time the websocket orderbook data drifts further and further from actual prices and becomes unusable. In fact, it can cause people to loose a lot of money. Is Bitmex team aware of this issue? If so, what is their suggested solution?

[CoreyNelson]

I'm not using Python but I'm wondering if this issue is actually caused by #273.

[bretton]

PS: Depending on the index.js content I'm receiving two different errors:
TypeError: Cannot read property 'map' of undefined at clone (/api-connectors/official-ws/nodejs/index.js:231:15) ....

I'm having the same issue. The problem appears to be on Bitmex side. The response is either

{}

or

[]

which is the valid blank response.

The bitmex connector dies on the {} reply for some reason giving the errors. If Bitmex fix the feed like it was for a few hours today, it'll work again.

Does anyone have another solution?

[mjzarrin]

I have the same problem. It has more than 1000 $ divergence on (XBT/USD) after 24 hours!

[ghost]

Oh lord, I just came from here: #310
Why does Bitfinex provide such a smooth wss and we're having such basic issues with BitMex. I would have thought it should be the other way around =/
So yeah +1 on the Delta issue

[surGeonGG]

@asiva It works perfectly when i remove the l2orderbook. Thanks!

That might be why the nodejs server works fine as well, since it doesn't subscribe to the l2orderbook by default if I'm not mistaken.

[mrbreaks]

this error still there
where is the bitmex team fix it please

[k0rean-rand0m]

Yep, the error is still here, but for those, who want to use Python + WebSocket and have have face with this awful bug, here's something, that you can do:

import websockets
import asyncio
import json

async def capture_data():
    uri = "wss://www.bitmex.com/realtime?subscribe=instrument:XBTUSD"
    async with websockets.connect(uri) as websocket:
    while True:
        data = await websocket.recv()
        data = json.loads(data)
        print('\n', data)

asyncio.get_event_loop().run_until_complete(capture_data())

Docs:
https://www.bitmex.com/app/wsAPI
https://websockets.readthedocs.io/en/stable/

I've checked it and data was identical with the BitMEX web-interface. No delays at all.

[globophobe]

As identified by @k0rean-rand0m could be an issue with api-connectors Python websocket-client dependency.

Might be better to subclass BitMEXWebsocket in bitmex_websocket.py to use something like asyncio, etc.

[mellertson]

@k0rean-rand0m Thanks for posting that code! It's definitely very helpful! Will private data like orders, fills, and positions be secure if transferred from bitmex to a wss client using code like you suggested? I assume it would be, due to the URI being "wws://*", but just wanted to confirm. I suppose I could always confirm using a network sniffer. But, figured it might be quicker to ask. Thanks again!

P.S. A couple years ago I had issues connecting to HitBTC with the same symptoms described in this issue. I re-wrote the web-socket client a few times trying to fix it and eventually gave up. But, I do remember now that using the websockets library with asyncio worked without interruption. I had forgotten all about the websockets library. So cool that half a world a way you were there to point me in the right direction! Cheers mate!

[stabilus]

The delta-server also produces lags (not just the python websocket). I did quite a bit of testing and found these results:

Lag duration varies

• with the volume of information it has to load from bitmex
• with the strain I put on the delta server by polling it

I created an endless loop polling the trade data (http://localhost:4444/trade?), selecting the last 10 seconds of data and calculating the volumes (in panda dataframes). Not applying any pause to the loop, I got about 10 polls/second. This was manageable during low volume periods. Comparing the timestamp of the latest trade with the UTC timestamp I got a delay of typically less than 1 second.

However during periods of higher volumes (number of trades / second > 20) the lag time started to increase - the higher the number of trades, the faster the increase of the lag.
Once the number of transactions decreased to below ca. 20/s, the lag started to recover and ended up back under a second.

Then I built a pause (say 0.1 s) into the loop. That yielded ca 5 polls / s. I found that there was far less lag. This loop could handle approx. 100 trades/s without building lag. It would increase to 2 seconds but quickly go back to below 1 s.

Also, straining the delta server with polls from a different app that ran in parallel influenced the lag negatively.

To conclude: The delta server needs "room to breath". It needs some processing power to load the data from bitmex. If you overload your server with too many polls it falls behind and you get a lag that increases in response to the data volume it needs to handle.

So, I ended up using a lag measurer and dynamically adapting my polling rate to the lag. Works for me.

[globophobe]

BitMEX has a REST API, which is nice.

If the client disconnects, wait for reconnect. Buffer the incoming data, while making a request to the REST API. Finally, filter the buffered data for missing trdMatchID from the REST API.

[janisdombr]

I try to collect 'orderBookL2' data but while every 10 seconds it's about 5-10% of messages with error because can't find id with that price. It occurs because many messages with action "delete" came before "update" with the same id. I thought that it's because of missing "insert" and I use your approach with asyncio lib but then I realised that if I ignore "delete" messages all is ok. But messages of 'orderBookL2' have no attribute 'timestamp'. Maybe you know how to solve it? I can update 'size' to 0 instead deleting row but it can be wrong way...

Yep, the error is still here, but for those, who want to use Python + WebSocket and have have face with this awful bug, here's something, that you can do:

import websockets
import asyncio
import json

async def capture_data():
    uri = "wss://www.bitmex.com/realtime?subscribe=instrument:XBTUSD"
    async with websockets.connect(uri) as websocket:
    while True:
        data = await websocket.recv()
        data = json.loads(data)
        print('\n', data)

asyncio.get_event_loop().run_until_complete(capture_data())

Docs:
https://www.bitmex.com/app/wsAPI
https://websockets.readthedocs.io/en/stable/

I've checked it and data was identical with the BitMEX web-interface. No delays at all.

[mandri87]

Yep, the error is still here, but for those, who want to use Python + WebSocket and have have face with this awful bug, here's something, that you can do:

import websockets
import asyncio
import json

async def capture_data():
    uri = "wss://www.bitmex.com/realtime?subscribe=instrument:XBTUSD"
    async with websockets.connect(uri) as websocket:
    while True:
        data = await websocket.recv()
        data = json.loads(data)
        print('\n', data)

asyncio.get_event_loop().run_until_complete(capture_data())

Docs:
https://www.bitmex.com/app/wsAPI
https://websockets.readthedocs.io/en/stable/

I've checked it and data was identical with the BitMEX web-interface. No delays at all.

I'have the same issue, data are delayed and I need a solution to get it in real time. I want to implement your proposal but I can't understand where I should put this piece of code in bitmex_websocket.py Could you or someone else please help me? Thanks in advance

[globophobe]

Much recommended https://github.com/bmoscon/cryptofeed

[tuncel3]

I have same problem with binance websocket aggregate trades stream. When market volatility increase, time stamp of websocket received data deviates from actual time.

I think it is because of client computer CPU speed. When there are many trades happening in volatile times, it becomes harder for client computer to handle so many "on message" interrupts and it queues them. After queue finishes, for example market volatility is reduced, it becomes normal.

I think the solution is to use a faster CPU on client side. Someone said he uses threading to receive data, if CPU doesn't have extra threads, python extra threads will not behave like extra threads.