Cancellation of Device subscription hangs following asynchronous completion or error

[glopesdev]

We found a weird edge-case where the current implementation of the Device observable sequence hangs indefinitely following asynchronous termination. To reproduce:

Example workflow source

<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.8.0-alpha018"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:harp="clr-namespace:Bonsai.Harp;assembly=Bonsai.Harp"
                 xmlns:rx="clr-namespace:Bonsai.Reactive;assembly=Bonsai.Core"
                 xmlns="https://bonsai-rx.org/2018/workflow">
  <Workflow>
    <Nodes>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="harp:Device">
          <harp:OperationMode>Active</harp:OperationMode>
          <harp:OperationLed>On</harp:OperationLed>
          <harp:DumpRegisters>true</harp:DumpRegisters>
          <harp:VisualIndicators>On</harp:VisualIndicators>
          <harp:Heartbeat>Disabled</harp:Heartbeat>
          <harp:IgnoreErrors>false</harp:IgnoreErrors>
          <harp:PortName>COM5</harp:PortName>
        </Combinator>
      </Expression>
      <Expression xsi:type="rx:PublishSubject">
        <Name>Events</Name>
      </Expression>
      <Expression xsi:type="SubscribeSubject">
        <Name>Events</Name>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="rx:Timer">
          <rx:DueTime>PT1S</rx:DueTime>
          <rx:Period>PT0S</rx:Period>
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="rx:Sample" />
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="rx:Take">
          <rx:Count>1</rx:Count>
        </Combinator>
      </Expression>
      <Expression xsi:type="WorkflowOutput" />
    </Nodes>
    <Edges>
      <Edge From="0" To="1" Label="Source1" />
      <Edge From="2" To="4" Label="Source1" />
      <Edge From="3" To="4" Label="Source2" />
      <Edge From="4" To="5" Label="Source1" />
      <Edge From="5" To="6" Label="Source1" />
    </Edges>
  </Workflow>
</WorkflowBuilder>

We were able to reproduce the blocking behavior both with OnCompleted and OnError. In this example, the workflow terminates on the first sample from Timer. Crucially, the termination is asynchronous with Device events. The observed behavior is that the workflow terminates but cancellation of the subscription hangs indefinitely (i.e. the workflow running state will never change back to Ready).

On investigation we found it is the call to the Dispose method of the underlying serial port object that hangs. Calling dispose on the serial port from a separate thread is sufficient to resolve the issue, and the serial port will then be successfully closed (and can be reopened again).

The problem with the asynchronous disposal logic is that it changes the currently synchronous semantics of serial port termination. Since disposal is not really synchronous there is a chance the serial port is still open if we either have a Repeat, Retry or restart the workflow quickly.