Observable behaves differently when wrapped

[OliverJAsh]

Bug Report

Current Behavior
Given this code:

import { Observable, of, OperatorFunction, timer } from 'rxjs';
import { catchError, concat, mergeMap, mergeMapTo, startWith } from 'rxjs/operators';

enum State {
    loading = 'loading',
    error = 'error',
}

const ajax$ = new Observable<State>(subscriber => {
    setTimeout(() => {
        subscriber.error('foo');
        subscriber.complete();
    }, 3000);
}).pipe(
    catchError(_error => of(State.error)),
    startWith(State.loading),
);

const delayedRepeatWhenError = (): OperatorFunction<State, State> => state$ =>
    state$.pipe(
        mergeMap(state => {
            const currentState$ = of(state);
            return state === State.error
                ? currentState$.pipe(concat(timer(3000).pipe(mergeMapTo(state$))))
                : currentState$;
        }),
    );

ajax$.pipe(delayedRepeatWhenError()).subscribe(console.log);

The output is as expected:

loading
error
loading
error

However, if the observable is wrapped and then subscribed to, the output is different and not as expected.

Code:

const wrappedAjax$ = new Observable<State>(subscriber => {
    const subscription = ajax$.subscribe(subscriber);
    return () => {
        subscription.unsubscribe();
    };
});

wrappedAjax$.pipe(delayedRepeatWhenError()).subscribe(console.log);

Output:

loading
error

This seems to be a regression from one of the releases after 6.2.2, as that's what I just upgraded from.

Reproduction

• REPL or Repo link: https://stackblitz.com/edit/typescript-bcrr1f?file=index.ts

[cartant]

Thanks for the repro of the bug. I've confirmed that this is due to #3963 (which was a refactor of #2457).

[cartant]

The problem appears to be related to the returning of a teardown function instead of returning the subscription itself.

If the observable is wrapped without the intermediate teardown function, it works fine:

const wrappedAjax$ = new Observable<State>(subscriber => ajax$.subscribe(subscriber));

Wrapping the unsubscribe call in a teardown function see this test pass and the subscription's unsubscription is added to itself. Clearly, that test is inadequate and an alternative mechanism for handling this is required.

[cartant]

I have a solution for this. I will submit a PR tomorrow.