order gets lost

[disaster123]

Hello,

i'm using the following code to place an order:

       trade = Ibkr2.myib.placeOrder(contract, order)
       # hack from: https://stackoverflow.com/questions/62248078/need-to-use-ib-sleep-to-send-multiple-orders-using-ib-insync
       Ibkr2.myib.sleep(0)

       # no need to wait until order is completely fullfilled
        orderok = False
        for c in range(20):
            print("IBKR: wait for trade: %s" % (trade))
            if (trade.orderStatus):
                print("IBKR: wait for orderstatus.status: %s" % (trade.orderStatus.status))
            if trade.orderStatus and trade.orderStatus.status and (trade.orderStatus.status == 'Cancelled' or trade.orderStatus.status == 'PreSubmitted' or 
                    trade.orderStatus.status == 'Submitted' or trade.orderStatus.status == 'Filled' or trade.orderStatus.status == 'PendingSubmit'):
                orderok = True
                break
            Ibkr2.myib.sleep(0)

While the output is:

IBKR: wait for trade: Trade(contract=CFD(conId=143916318, symbol='EUR', exchange='SMART', currency='USD', localSymbol='EUR.USD', tradingClass='EUR.USD'), order=LimitOrder(orderId=1761, clientId=999, action='SELL', totalQuantity=2190000, lmtPrice=1.1145, tif='GTC', orderRef=77263676), orderStatus=OrderStatus(orderId=1761, status='PendingSubmit', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[],
...

TradeLog at the same time:
log=[TradeLogEntry(time=datetime.datetime(2024, 8, 22, 6, 21, 39, 305530, tzinfo=datetime.timezone.utc), status='PendingSubmit', message='', errorCode=0)], advancedError='')

This order does not show up in ibkr client and was never executed nor is it in pending orders. This happens frequently...

Greets,

[praditik]

Hi @disaster123 ,

I do not see any obvious problems with the script you shared and my experience says it should work fine. A few steps you could use to debug are

1. Use errorEvent and connect it to your local function that can be used to print the errors/messages from IBKR. This might show details of what might be going wrong in the order.
2. Sometimes, if order is invalid or the contract is invalid, the order might be rejected. So, try checking that.

Would be helpful if you can share more relevant code and even more detailed logs.

Happy to help.

[disaster123]

Thx.
1.) implemented and will see what happens.
2.) not possible - exact same trade - gets executed fine just seconds before

[gnzsnz]

I would suggest to use events rather than a blocking loop right after placing your orders.

the reason you need to sleep(0) is to trigger the event loop, but AFAIK there is no warranty that it will pick up what you want. by using an even you are notified when it happens. what you are doing now is blocking and not allowing the event loop to run

Something like this should work

def on_fill(trade,fill):
  orderok = True
  print("IBKR: orderstatus.status: %s" % (trade.orderStatus.status))

orderok = False
trade = Ibkr2.myib.placeOrder(contract, order)
trade.fillEvent.connect(on_fill)

notice that the ordering notebook is using

trade = ib.placeOrder(contract, order)
while not trade.isDone():
    ib.waitOnUpdate()

which does not block the event loop

Always follow The One Rule

[disaster123]

@gnzsnz thanks for your suggestions and help. Some questions.

trade = Ibkr2.myib.placeOrder(contract, order)
trade.fillEvent.connect(on_fill)

isn't a race possible in this case? - so the order is filled before the event is attached? Also Submitted is OK to me as well.

I had waitOnUpdate used in the past but there were problems with it - for example that if there i no event the loop neber proceed. For example trade goes to Submitted and than nothing happens - your loop blocks.

[mattsta]

A couple things can happen:

I've seen orders go directly from PreSubmitted to Filled with nothing else in the middle.

You can subscribe to also just trade.statusEvent and receive all status updates (including Submitted/Modified/Cancelled/Filled).

There is currently a bug where if you submit an order modification which is rejected (update has a bad price or rejected algo settings), the trade gets deleted from your client and won't show up until a restart even though it's still live (ValidationError status updates). This has a fix coming eventually.

After you place your trade, the trade status and event handlers can't run again until you wait with a proper sleep or another await entry point. So it's fine to run trade = placeOrder(); trade.statusEvent += eventHandler because only your code is running between those two statements and no other updates can happen.

Also, it all depends on your own system design and program logic. You should also subscribe your entire ib client to status and error events (also documented at the usual place. If your trade fails to work the first time, it won't exist for any status updates. You probably want ib = IB(); ib.orderStatusEvent += globalOrderStatusEventHandler to receive updates for any order changes.

None of this is simple or easy to get right without a lot of testing in various live situations, so just keep trying on maybe a test/sandbox account or low value trades until it starts working as expected.

[gnzsnz]

it's possible that the order is filled before, but I have never seen it to actually happen. on my code i use transmit=False, then i attach the event, then submit again with transmit=True, but because my trade logic depends on this and i want to be 300% sure. But i never use it on my "quick tests" because it just never happens.

not sure what could be your problem with waitOnUpdate, personally I never use it. take into account that if you code is blocking, is possible that you are blocking waitOnUpdate as well, that's why i suggested The One Rule.

the original name of this project was great, because it's "insync". from the surface it looks synchronous, but it's just an appearance.

[mattsta]

i attach the event, then submit again with transmit=True,

That's actually an interesting use case. Maybe we could allow the order API to also take an empty Trade object it populates itself, so you could set event handlers on a Trade object before the trade is submitted. You can also always listen to the global ib.* events instead of per-task events too if per-task events can't be caught directly.

Given the way the event loop works, between placeOrder() and the trade value being returned, nothing else happens, so it's impossible for events to trigger between those two cases (unless you yield to another await in the middle).

Another issue with the event system is it doesn't queue events. Events are either fired against active listeners or dropped forever. We may want to eventually explore allowing certain events to "queue up" until consumed by an event handler (I currently get around this by having my event handlers set their own asyncio.Event() update signals the event loop can wait on too, and this also allows me to pre-filter event notifications like when an update triggers Submitted->Submitted and I don't care to check the status because nothing about the execution changed).

[gnzsnz]

we could add a parameter

async def trade_fill_event_handler():
   your code here

trade = ib.placeOrder(contract,oder,fillEvent=trade_fill_event_handler)

but to be honest, they way it is right now is fine with me.