Meta-IRQs making cooperative threads preemptive #20255
Comments
|
Is the expected behavior here that if B is running, IRQ M fires and releases a resource that A was sleeping on, that context returns back to B instead of A? |
|
Yes, returning to B (as a regular IRQ would do as described here) would be the expected behavior. |
aescolar
added
priority: high
|
Raising to high following @Vudentz advice |
|
I believe that the issue could be solved in a rather simple way for the single-processor case. Upon switching from a cooperative thread to a meta-IRQ, a variable could remember which thread was preempted. That thread would then be the one to switch to once a non-meta-IRQ is to be scheduled. For the SMP case, it is not clear to me what the extected behavior for the preempted cooperative thread is. Should it remain on the same CPU? Would it be allowed to migrate? |
|
Indeed, a code fix isn't particularly hard. We'd need a per-CPU tracking of the "last preempted cooperative thread", and on exit from a metairq stack just make that current (if it's still runnable, of course). The SMP case of migration is an interesting edge case: I'd say no, we don't want to make it schedulable, because the reason it was cooperative in the first place was probably for locking and we don't want it to race with other threads. And there are some other edge cases: these threads can be suspended, reach a timeout or be killed while they are preempted and we need to track that. FWIW: my original hope was that we could just fix this with documentation. Cooperative threading was really only intended for locking, it doesn't work for interrupt bottom half functionality (where it should be preemptible by higher priority threads -- really that's why MetaIRQ exists in the first place) and it doesn't provide locking at all in SMP (where knowing you won't be preempted says nothing about what other CPUs will run). But @aescolar points out we have subsystems relying on cooperative behavior that will break if used with a metairq, so I guess a code fix is needed. |
|
My attempt at a fix is posted above, sorry for crossing with the other work. It passes the test in @wopu-ot 's PR, but it does not include it (can you resubmit that as a separate patch please?). Pushing for early review, it hasn't passed a full sanitycheck for me yet. |
When a MetaIRQ preempts a cooperative thread, that thread would be added back to the generic run queue. When the MetaIRQ is done, the highest priority thread will be selected to run, which may obviously be a cooperative thread of a higher priority than the one that was preempted. But that's wrong, because the original thread was promised that it would NOT be preempted until it reached a scheduling point on its own (that's the whole point of a cooperative thread, of course). We need to track the thread that got preempted (one per CPU) and return to it instead of whatever else the scheduler might have found. Fixes zephyrproject-rtos#20255 Signed-off-by: Andy Ross <[email protected]>
When a MetaIRQ preempts a cooperative thread, that thread would be added back to the generic run queue. When the MetaIRQ is done, the highest priority thread will be selected to run, which may obviously be a cooperative thread of a higher priority than the one that was preempted. But that's wrong, because the original thread was promised that it would NOT be preempted until it reached a scheduling point on its own (that's the whole point of a cooperative thread, of course). We need to track the thread that got preempted (one per CPU) and return to it instead of whatever else the scheduler might have found. Fixes #20255 Signed-off-by: Andy Ross <[email protected]>
Describe the bug
Meta-IRQs can make cooperative threads preempt each other. The documentation on meta-IRQs mentions that "this breaks the promise made to cooperative threads" but it is unexpected that this implies not only that meta-IRQs can preempt cooperative threads, but also that cooperative threads can preempt each other.
To Reproduce
Consider 3 threads with decreasing priorities { M, A, B }, where M is a meta-IRQ and A and B are cooperative threads. Furthermore, consider that A is waiting on a semaphore and B is executing when M is activated and preempts B. M gives the semaphore to A and exits. This causes A to be scheduled as soon as M exits, effectively preempting B. A and B are no longer cooperative with regard to each other. See metairq.c for an example to trigger the issue.
Expected behavior
Cooperative threads do not preempt each other.
Impact
Meta-IRQs cannot be used in cases where they would make sense, e.g., to implement Mayflies in the Bluetooth stack. This can be worked around, but the remaining use cases for meta-IRQs are quite limited.
The text was updated successfully, but these errors were encountered: