Beyond Speech-to-Text

[AdamSobieski]

With respect to post-text speech recognition (e.g. speech-to-SSML, speech-to-hypertext, speech-to-X₁), we can consider:

from:

// Item in N-best list
[Exposed=Window]
interface SpeechRecognitionAlternative {
    readonly attribute DOMString transcript;
    readonly attribute float confidence;
};

to:

// Item in N-best list
[Exposed=Window]
interface SpeechRecognitionAlternative {
    readonly attribute object transcript;
    readonly attribute float confidence;
};

then client-side, server-side or third-party components or services could return either text or XML content per recognition result. That is, transcript could be either a DOMString or a DOMElement.

Speech-to-text is too lossy. Information pertaining to prosody, intonation, emphases and pauses are discarded in text-formatted output. Such information can be useful, for instance, in informing machine translation components and services.

[AdamSobieski]

Or might this be a DataTransfer scenario?

as per:

// Item in N-best list
[Exposed=Window]
interface SpeechRecognitionAlternative {
    readonly attribute DataTransfer transcript;
    readonly attribute float confidence;
};

and perhaps (see also #10 , #37):

[Exposed=Window,
  Constructor,
  Constructor(DOMString text)]
interface SpeechSynthesisUtterance : EventTarget {
    attribute DataTransfer text;
    attribute DOMString lang;
    attribute SpeechSynthesisVoice? voice;
    attribute float volume;
    attribute float rate;
    attribute float pitch;

    attribute EventHandler onstart;
    attribute EventHandler onend;
    attribute EventHandler onerror;
    attribute EventHandler onpause;
    attribute EventHandler onresume;
    attribute EventHandler onmark;
    attribute EventHandler onboundary;
};