Directline speech - web socket ends with 200 response #3407

collegewap · 2020-08-12T06:48:01Z

Screenshots

Version

4.9.1

Describe the bug

I have enabled directline speech in our bot by using the following example and I am getting the error "Error during WebSocket handshake: Unexpected response code: 200". I tried both the options of passing token and passing the subscription key and facing the same issue.

https://github.com/microsoft/BotFramework-WebChat/tree/master/samples/03.speech/a.direct-line-speech

Note: Cognitive speech approach works fine.

What could be the possible cause for this?

Zerryth · 2020-08-12T21:57:36Z

I have enabled directline speech in our bot

@collegewap it seems like there might be something wrong with how you configured directline speech for your bot, as for a first test, I just simply did the following successfully:

Test with the Already-Made Mockbot Sample

(mockbot is the bot that the webchat sample talks to, whose code is separate from the 03.speech/a.direct-line-speech sample)

git clone https://github.com/microsoft/BotFramework-WebChat.git
npx serve
Navigate to http://localhost:5000/
Click the microphone icon and verbally say "hello"
Mockbot replies back with audio

No idea how you got the error that you did, as you didn't provide repro steps as to what you did exactly to set up your bot...however I'll provide repro steps below on how to configure speech in your bot successfully.

Setting up a Bot Configured to Direct Line Speech

This is just setting up an echo bot with DLS to enable speaking to your bot. It can be succinctly summarized as

Set up a bot in Azure that has Direct Line Speech channel enabled
Run sample 03.speech/a.direct-line-speech (npx serve) and speak to your DLS-enabled bot

Get a fresh clone of Echo Bot and Test Locally

git clone https://github.com/microsoft/BotBuilder-Samples.git
Navigate to 02.echo-bot
Verify that you can chat with the bot locally (npm i, npm start, open Bot Framework Emulator and just type a message and receive an echo--it's currently just a plain Echo Bot with no speech enabled yet)

Deploy your bot to Azure

You can deploy your bot many ways, however one way is via zip-deployment using AZ CLI
Once your bot is deployed to Azure, verify that it's working when you go to Bot Channels Registration > Test in Web Chat:

Note: I had just edited my bot before deploying to say "Speech Echo" so it's obvious that I deployed the right bot properly
Additionally make sure once you have your Bot's AppId and Password, to have it in your .env file that's deployed to Azure (might need to redploy, depending on the order that you did things, which you can do so easily using az webapp deployment source config-zip --resource-group "<resource-group-name>" --name "<name-of-web-app>" --src "<project-zip-path>")

Provision Speech cognitive service

Search the Marketplace for "Speech" and create a Speech cognitive service
Save one of the keys, which you'll use in your WebChat later

Enable DirectLine Speech Channel

In Azure, go to your App Service > Configuration under "Settings" on the left-hand side > For "Web sockets" select "on" > Save:
Once again in Azure, go to Bot Channels Registration > Channels > Direct Line Speech
Select the Speech cognitive service you created earlier (I believe you can only link one speech service to one bot at a time, fyi) > Save
Bot Channels Registration > Settings > check Enable Streaming Endpoint > Save

Register Direct Line Speech Channel

Run sample 03.speech/a.direct-line-speech and talk to the DLS-enabled bot.

Add your Speech cognitive service key and region to the HTML file for sample 3a

<!DOCTYPE html>
<html lang="en-US">
  <head>
    <title>Web Chat: Using Direct Line Speech</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <!-- Direct Line Speech adapter set is only available in full bundle -->
    <script crossorigin="anonymous" src="https://cdn.botframework.com/botframework-webchat/latest/webchat.js"></script>
    <style>
      html,
      body {
        height: 100%;
      }

      body {
        margin: 0;
      }

      #webchat {
        height: 100%;
        width: 100%;
      }
    </style>
  </head>

  <body>
    <div id="webchat" role="main"></div>
    <script>
      (async function() {
        const adapters = await window.WebChat.createDirectLineSpeechAdapters({
          fetchCredentials: {
            region: 'xxxxx',
            subscriptionKey: '89exxxxxxxxxxxxxxxxxxxxxxxxxxxx2'
          }
        });

        console.log(adapters);

        // Pass the set of adapters to Web Chat.
        window.WebChat.renderWebChat(
          {
            ...adapters
          },
          document.getElementById('webchat')
        );

        document.querySelector('#webchat > *').focus();
      })().catch(err => console.error(err));
    </script>
  </body>
</html>

npx serve
Navigate to localhost:5000, click the microphone, speak to the bot

The bot speaks back!

If you follow the steps above, which are similar to this tutorial on enabling speech in your bot (they use C#), then you should be able to run sample 3a just fine

Zerryth · 2020-08-12T23:11:08Z

Hmm, looking around there does seem to be some issue on the other layers regarding DLS.
microsoft/cognitive-services-speech-sdk-js#221 (comment)

What kind of browser and OS are you using @collegewap ?

collegewap · 2020-08-13T03:55:26Z

@Zerryth

The bot is working fine (Tried example 1a). If I try the speech services also, it works fine. I made sure that all the steps to enable directline speech are done correctly. Still facing the issue.

I am using Windows 10+ Chrome. Tried in Firefox as well. Same error. I tried https://compulim.github.io/webchat-loader/ with the mock bot and getting the same error.

My region is eastus2.

Zerryth · 2020-08-13T04:27:44Z

Hmm...ok, @collegewap, I think what we need now here is painfully explicit step-by-step of what you're doing, just so that we can get a repro of your issue.

Please provide exact repro steps of at least one thing you're doing to get your error, so that we could hopefully get it too and dig down to a resolution

Because when I try going to, for example:

webchat-loader > "[Public] MockBot (Direct Line Speech)" > "Open Web Chat in a new window"

I never get the "Error during WebSocket handshake: Unexpected response code: 200" that you're unfortunately facing 😞

There are probably discrepancies between what you're doing and what we're doing, and it's not clear what they are without more detail.

Curious to know what endpoint you're trying to call from WebChat to your bot, because looking at just this code snippet from sample 3a

          const res = await fetch('https://webchat-mockbot-streaming.azurewebsites.net/speechservices/token', {
            method: 'POST'
          });

To get a token for mockbot, it calls https://webchat-mockbot-streaming.azurewebsites.net/speechservices/token, because in mockbot it has a token server set up, however trying to call https://<YOUR-OWN-HOSTNAME>.azurewebsites.net/speechservices/token will not succeed if your own bot doesn't have that route built in to handle such call.

Anyways, so again, please provide as much detail as possible, so we can get a repro on the error

collegewap · 2020-08-13T05:15:06Z

Here are the steps I am doing:

Open https://compulim.github.io/webchat-loader/
Select "MockBot (Direct Line Speech)"
Clear the generated token
Choose "east us 2" as the region.
Provide my speech key and click on "Generate Token".
Click on "Open Web Chat in a new window"
Click on the mic button and I would see the above-mentioned error.

My setup:

And about 3a, we have an endpoint that generates the token. Also, I have tried the example given by you in #3407 (comment), by passing the speech key and getting the same error.

collegewap · 2020-08-13T09:50:20Z

@Zerryth We followed the same steps in another instance and it worked fine. Not sure what is the issue with this specific bot instance. Thanks for your help.

collegewap added Bot Services Required for internal Azure reporting. Do not delete. Do not change color. bug Indicates an unexpected problem or an unintended behavior. customer-reported Required for internal Azure reporting. Do not delete. labels Aug 12, 2020

carlosscastro added the support Internal request for assistance from the Support Team label Aug 12, 2020

jwiley84 assigned Zerryth Aug 12, 2020

Zerryth added the customer-replied-to Required for internal reporting. Do not delete. label Aug 12, 2020

collegewap mentioned this issue Aug 13, 2020

WebSocket connection immediately closed on macOS microsoft/cognitive-services-speech-sdk-js#221

Closed

collegewap closed this as completed Aug 13, 2020

Zerryth mentioned this issue Oct 27, 2020

Verify if Streaming bots can call skills microsoft/botbuilder-js#2918

Closed

Zerryth mentioned this issue Feb 3, 2021

WebSocket connection failed: Error during WebSocket handshake: Unexpected response code: 400 #3691

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Directline speech - web socket ends with 200 response #3407

Directline speech - web socket ends with 200 response #3407

collegewap commented Aug 12, 2020

Zerryth commented Aug 12, 2020 •

edited

Loading

Zerryth commented Aug 12, 2020

collegewap commented Aug 13, 2020 •

edited

Loading

Zerryth commented Aug 13, 2020 •

edited

Loading

collegewap commented Aug 13, 2020

collegewap commented Aug 13, 2020

Directline speech - web socket ends with 200 response #3407

Directline speech - web socket ends with 200 response #3407

Comments

collegewap commented Aug 12, 2020

Screenshots

Version

Describe the bug

Zerryth commented Aug 12, 2020 • edited Loading

Test with the Already-Made Mockbot Sample

Setting up a Bot Configured to Direct Line Speech

Zerryth commented Aug 12, 2020

collegewap commented Aug 13, 2020 • edited Loading

Zerryth commented Aug 13, 2020 • edited Loading

collegewap commented Aug 13, 2020

collegewap commented Aug 13, 2020

Zerryth commented Aug 12, 2020 •

edited

Loading

collegewap commented Aug 13, 2020 •

edited

Loading

Zerryth commented Aug 13, 2020 •

edited

Loading