Directline Speech Sample unsuccessful Directline WebSocket connection

[steps371]

Screenshots

Version

I'm using the 3. Speech example in the BotFramework-WebChat npm package.

Here are the versions of the npm packages I'm running:

<meta name="botframework-directlinespeech:version" content="4.9.1">
<meta name="botframework-webchat:bundle:variant" content="full">
<meta name="botframework-webchat:bundle:version" content="4.9.1">
<meta name="botframework-webchat:core:version" content="4.9.1">
<meta name="botframework-webchat:ui:version" content="4.9.1">

Describe the bug

Our team has been struggling to activate directline speech in a webchat in our react examples, which prompted us to reproduce the error in a barebones bot-framework example.

When adjusting the speech example to connect to a deployed bot instead of the mockbot, I receive an error (in the console) when clicking on the microphone to record speech, and the webchat freezes up.

Steps to reproduce

1. Download BotFramework-WebChat repo
2. Replace the the code in between <script> tags in 03.speech example with the following:

(async function() {
        const fetchCredentials = async () => {
            const res = await fetch("https://westus.api.cognitive.microsoft.com/sts/v1.0/issueToken", {
            headers: {"Ocp-Apim-Subscription-Key": "XXXX"},
            method: "POST"
            })

          if (res.status === 200) {
          const region = 'westus'
          const authorizationToken = await res.text()
          return {region, authorizationToken}
          } else {
          return (new Error('error!'))
          }

          return { authorizationToken, region };
        };

        // Create a set of adapters for Web Chat to use with Direct Line Speech channel.
        // Please refer to /packages/directlinespeech/src/createAdapters.js for list of supported options.
        const adapters = await window.WebChat.createDirectLineSpeechAdapters({
          fetchCredentials
        });

        // Pass the set of adapters to Web Chat.
        window.WebChat.renderWebChat(
          {
            ...adapters,          },
          document.getElementById('webchat')
        );

        document.querySelector('#webchat > *').focus();
      })().catch(err => console.log({err}));

3. Run example with npx serve, navigate to localhost:5000 and you should see this:

4. After clicking on the microphone, the webchat hangs on the following, and the following error is logged in the console

WebSocket connection to 'wss://westus.convai.speech.microsoft.com/api/v3?language=en-GB&format=simple&Authorization=XXXX-ConnectionId=XXX’ failed: Error during WebSocket handshake: Unexpected response code: 200

And in the network tab (visible even before pressing the microphone):

Expected behavior

Following the directline speech webchat example, I would've expected recorded speech to be transcribed to text, and sent to the bot.

Additional context

The speech resource and deployed bot works when configured on the bot-framework emulator, so the issue is likely not with the provisioned resources.

[Bug]

[compulim]

I copied your code snippets into our 03.speech/a.direct-line-speech sample by replacing the fetchCredentials part and it works.

1. What browsers and OS are you testing it on?
2. If on Windows, can you use Fiddler and see what's the HTTP response body on the request?
   • Browser network tab may not give you this information, you might need to use Fiddler instead

[EricDahlvang]

@steps371 please respond to William's questions. We would like to close this in a few days.

[steps371]

Apologies for the delay in response

I experienced this issue on Mac Catalina in Both Chrome and Safari browsers. I believe a colleague was able to get the sample working on Windows.

Will investigate further tomorrow

[steps371]

Still running into the same error.

OS:
macOS Catalina 10.15.4

Browsers:
Google Chrome Version 84.0.4147.89 (Official Build) (64-bit)
Safari Version 13.1 (15609.1.20.111.8)

[steps371]

Does anyone have any suggestions on what the possible issue might be? Still stuck on this

[peterinnesmsft]

@compulim can you take a look and help with @steps371's latest question?

[compulim]

@steps371 do you think this is dupe of microsoft/cognitive-services-speech-sdk-js#221?

[steps371]

@compulim, definitely.

[pavolum]

This is a duplicate and is being tracked here: microsoft/cognitive-services-speech-sdk-js#221