intelligence-SpeechToText

Speech To Text Bot Sample

A sample bot that illustrates how to use the Microsoft Cognitive Services Bing Speech API to analyze an audio file and convert the audio stream to text.

Prerequisites

The minimum prerequisites to run this sample are:

• The latest update of Visual Studio 2015. You can download the community version here for free.
• The Bot Framework Emulator. To install the Bot Framework Emulator, download it from here. Please refer to this documentation article to know more about the Bot Framework Emulator.
• [Recommended] Visual Studio Code for IntelliSense and debugging, download it from here for free.
• This sample currently uses a free trial Microsoft Cognitive service key with limited QPS. Please subscribe to Bing Speech Api services here and update the MicrosoftSpeechApiKey key in key in Web.config file to try it out further.

Usage

Attach an audio file (wav format).

Code Highlights

Microsoft Cognitive Services provides a Speech Recognition API to convert audio into text. Check out Bing Speech API for a complete reference of Speech APIs available. In this sample we are using the Speech Recognition API using the REST API.

In this sample we are using the API to get the text and send it back to the user. Check out the use of the MicrosoftCognitiveSpeechService.GetTextFromAudioAsync() method in the Controllers/MessagesController class.

var audioAttachment = activity.Attachments?.FirstOrDefault(a => a.ContentType.Equals("audio/wav"));
if (audioAttachment != null)
{
    using (var client = new HttpClient())
    {
        var stream = await client.GetStreamAsync(audioAttachment.ContentUrl);
        var text = await this.speechService.GetTextFromAudioAsync(stream);
        message = ProcessText(activity.Text, text);
    }
}

and here is the implementation of MicrosoftCognitiveSpeechService.GetTextFromAudioAsync() in Services/MicrosoftCognitiveSpeechService.cs

/// <summary>
/// Gets text from an audio stream.
/// </summary>
/// <param name="audiostream"></param>
/// <returns>Transcribed text. </returns>
public async Task<string> GetTextFromAudioAsync(Stream audiostream)
{
    var requestUri = @"https://speech.platform.bing.com/recognize?scenarios=smd&appid=D4D52672-91D7-4C74-8AD8-42B1D98141A5&locale=en-US&device.os=bot&version=3.0&format=json&instanceid=565D69FF-E928-4B7E-87DA-9A750B96D9E3&requestid=" + Guid.NewGuid();

    using (var client = new HttpClient())
    {
        var token = Authentication.Instance.GetAccessToken();
        client.DefaultRequestHeaders.Add("Authorization", "Bearer " + token.access_token);

        using (var binaryContent = new ByteArrayContent(StreamToBytes(audiostream)))
        {
            binaryContent.Headers.TryAddWithoutValidation("content-type", "audio/wav; codec=\"audio/pcm\"; samplerate=16000");

            var response = await client.PostAsync(requestUri, binaryContent);
            var responseString = await response.Content.ReadAsStringAsync();
            try
            {
                dynamic data = JsonConvert.DeserializeObject(responseString);
                return data.header.name;
            }
            catch (JsonReaderException ex)
            {
                throw new Exception(responseString, ex);
            }
        }
    }
}

Outcome

You will see the following when connecting the Bot to the Emulator and send it an audio file and a command:

Input:

"What's the weather like?"

Output:

More Information

To get more information about how to get started in Bot Builder for .NET and Microsoft Cognitive Services Bing Speech API please review the following resources:

• Bot Builder for .NET
• Microsoft Cognitive Services Bing Speech API