W3C

Speech Recognition Grammar Specification
for the W3C Speech Interface Framework

W3C Working Draft 20 August 2001

This version:
http://www.w3.org/TR/2001/WD-speech-grammar-20010820
Latest version:
http://www.w3.org/TR/speech-grammar
Previous version:
http://www.w3.org/TR/2001/WD-speech-grammar-20010103
Editors:
Andrew Hunt, SpeechWorks International
Scott McGlashan, PipeBeach
Contributors:
See Acknowledgements

Abstract

This document defines syntax for representating grammars for use in speech recognition so that developers can specify the words and patterns of words to be listened for by a speech recognizer. The syntax of the grammar format is presented in two forms, an augmented BNF syntax and an XML syntax. The specification intends to make the two representations directly mappable and allow automatic transformations between the two forms.

Status of this Document

This is the 20th August 2001 last call Working Draft of the "Speech Recognition Grammar Specification", and incorporates changes in response to feedback on the previous draft. This last call review period ends 28 September 2001. You are encouraged to subscribe to the public discussion list <www-voice@w3.org> and to mail in your comments before the review period ends. To subscribe, send an email to <www-voice-request@w3. org> with the word subscribe in the subject line (include the word unsubscribe if you want to unsubscribe). A public archive is available online.

This specification describes markup for grammars for use in speech recognition, and forms part of the proposals for the W3C Speech Interface Framework. This document has been produced as part of the W3C Voice Browser Activity, following the procedures set out for the W3C Process. The authors of this document are members of the Voice Browser Working Group (W3C Members only). Note that the name of the document has been changed from "grammar-spec" to "speech-grammar" to match how people have been referring to the specification.

To help the Voice Browser working group build an implementation report, (as part of advancing the document on the W3C Recommendation Track), you are encouraged to implement this specification and to indicate to W3C which features have been implemented, and any problems that arose.

This is a W3C Working Draft for review by W3C Members and other interested parties. It is a draft document and may be updated, replaced or made obsolete by other documents at any time. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress".

This is work in progress and does not imply endorsement by the W3C membership. A list of current W3C Recommendations and other technical documents, including Working Drafts and Notes, can be found at http://www.w3.org/TR/.

Table of Contents

1. Introduction

This document defines the syntax for grammar representation. The grammars are intended for use by speech recognizers and other grammar processors so that developers can specify the words and patterns of words to be listened for by a speech recognizer.

The syntax of the grammar format is presented in two forms, an Augmented BNF (ABNF) syntax and an XML syntax. The specification ensures that the two representations are semantically mappable to allow automatic transformations between the two forms.

Section 2, Section 3 and Section 4 define the ABNF and XML grammar formats. Readers may find the Examples in Appendix A instructive in understanding the specification. Section 5 defines the Conformance criteria for grammar documents and for grammar processors such as speech recognizers. Section 6 identifies a number of areas of Future Study for the grammar specification under consideration by the W3C Voice Browser Working Group.

This W3C Standard is known as the Speech Recognition Grammar Specification and is modelled on the JSpeech Grammar Format (JSGF) specification, which is owned by Sun Microsystems, Inc., California, U.S.A.

1.1 Grammar Processors and User Agents

A grammar processor is any entity that accepts as input grammars as described in this specification.

A user agent is a grammar processor that accepts user input and matches that input against a grammar to produce a recognition result that represents the detected input.

As the specification title implies, speech recognizers are an important class of grammar processor. Another class of grammar processor anticipated by this specification is a DTMF detector (see Section 4.1.3 and Appendix E). The type of input accepted by a user agent is determined by the mode or modes of grammars it can process: e.g. speech input for "voice" mode grammars and DTMF input for "dtmf" mode grammars.

For simplicity, through-out this document references to a speech recognizer apply to other types of grammar processor unless explicitly stated otherwise.

A speech recognizer is a user agent with the following inputs and outputs:

1.2 Scope

The primary use of a speech recognizer grammar is to permit a speech application to indicate to a recognizer what it should listen for, specifically:

Many speech recognizers also support the Speech Recognition N-Gram Grammar Specification. Both specifications define ways to set up a speech recognizer to detect spoken input but define the word and patterns of words by different and complementary means. Some recognizers permit cross-references between grammars in the two formats. The rule reference element of this specification describes how to reference an N-gram document.

The grammar specification does not address a number of other issues that affect speech recognition performance. Most of the following capabilities are addressed by the context in which a grammar is referenced or invoked: for example, through the Dialog Markup Language or through a Speech Recognizer API.

1.3 Grammar Conversions

The ABNF form and XML form and specified to ensure that the two representations are semantically mappable. It should be possible to automatically convert an ABNF grammar to an XML grammar (or the reverse) so that the semantic performance of the grammars are identical. Equivalence of semantic performance implies that:

  1. Both grammars accept the same language as input and reject the same language as input
  2. Both grammars parse any input string identically

The XSL Transformation document in Appendix F demonstrates automatic conversion from XML to ABNF. The reverse conversion requires an ABNF parser and a transformational program.

There are inherent limits to the automatic conversion to and from ABNF and XML forms.

Issue: has been noted that the statement above regarding entity references is the only statement on XML entities in the specification. It is assumed that an XML Grammar Processor must handle all XML entities as any XML processor would given the Conformance requirements in Section 5.4. In a subsequent draft this issue should be more effectively documented.

1.4 Semantic Interpretation

The W3C Voice Browser Working Group is currently developing the "Semantic Interpretation for Speech Recognition" specification. This forthcoming specification defines the language for the content of grammar tags and the transformation of spoken or other input in a semantic result.

To ensure grammar portability the working group plans to require that all compliant grammar processor must support the W3C semantic tag specification in conjunction with the grammar compliance requirements.

Readers should not assume that the simple tag examples in this document are indicative of the direction of the semantic tag specification. Examples will be updated once the first working draft of the "Semantic Interpretation for Speech Recognition" specification is published.

2. Rule Expansions

A rule expansion is a regular expression that defines patterns of tokens, rule references and combinations of these.

2.1 Token

A token (aka, a terminal symbol) is the part of a grammar that defines words or other entities that may be spoken. In both the XML and ABNF forms any unmarked text is a token.

The grammar format assumes that tokens can be resolved as lexicon entries and associated pronunciations by the recognizer. Grammar authors can improve document portability by avoiding characters and forms in tokens that do not have obvious pronunciations. For example, for English, punctuation should not be used within tokens but instead "spelled" versions should be used: "&" as "ampersand" or "and"; "+" as "plus"; "<" as "less than" or "open angle bracket"; and so on. A grammar processor should support digits (e.g. "0" though "9" for European scripts) but is not required to support natural numbers as tokens (e.g. "10", "1000", "4324", "3.1415").

Both the ABNF and XML forms permit tokens to be quoted to permit the token to contain white-space or other special symbols. White-space is not preserved when comparing tokens. Leading and trailing white-space in a quoted token is ignored. Any token-internal white-space is collapsed and is treated as if a single "space" character (#x20) was present. Thus the following are all equivalent tokens:

"San Francisco"
" San Francisco "
"San 
Francisco"
"   San    Francisco    "

Because the presence of whitespace within a token is significant the following are distinct tokens.

"San Francisco"
"SanFrancisco"
"San_Francisco"

A future draft of this specification will define semantic processing behavior. It is expected that white-space will be preserved in semantic results.

A future version of this specification may address more complex lexicon lookup issues including case sensitivity and other string comparison behaviors.

ABNF Form

Any plain text is a token. Tokens are delimited by white-space or by symbols with special syntactic function (e.g. ; = | * + <> () [] {} /* */ //). Tokens may be explicitly quoted if they contain white-space or special symbols. The language marker for a token (shown below for US English) is defined in Section 2.7

hello
bon voyage
this is a test    // sequence of four tokens
"San Francisco"
("San Francisco")!en-US
2
Special cases

An empty quoted token or a quoted token containing only whitespace is not legal.

// Illegal
""
"   "

XML Form

Follows the same style as the ABNF tokens including the same use of quotes. XML delimiters act as token delimiters.

As an alternative to quoting tokens, in the XML form a token may be explicitly delimited by a "token" element. The content of the element is PCDATA representing a single token. Section 2.7 defines the use of the "lang-list" attribute to indicate the language or languages of an expansion.

hello
bon voyage
this is a test    (sequence of four tokens)
"San Francisco"
<token lang-list="en-US">San Francisco</token>

2
Special cases

An empty quoted token or a quoted token containing only whitespace is not legal. An empty token element or a token element containing only whitespace is not legal.

// Illegal
""
"   "
<token/>
<token></token>
<token>  </token>

2.2 Rule Reference

Rulenames: Every rule definition has a local name that must be unique within the scope of the grammar in which it is defined. Legal rule names must be legal XML IDs as defined in the XML specification as the "Name" production in Section 2.3. Section 3.1 documents the rule definition mechanism and the legal naming of rules.

This table summarizes the various forms of rule reference that are possible within and across grammar documents.

Reference type ABNF Form XML Form
2.2.1: Local rule reference $rulename <ruleref uri="#rulename"/>
2.2.2: Reference to a named rule of a grammar identified by a URI $(grammarURI#rulename) <ruleref uri="grammarURI#rulename"/>
2.2.2: Reference to the root rule of a grammar identified by a URI $(grammarURI) <ruleref uri="grammarURI"/>
2.2.2: Reference to a named rule of a grammar identified by a URI with a Media Type $(grammarURI#rulename)~(mime-type) <ruleref uri="grammarURI#rulename" type="mime-type"/>
2.2.2: Reference to the root rule of a grammar identified by a URI with a Media Type $(grammarURI)~(mime-type) <ruleref uri="grammarURI" type="mime-type"/>
2.2.3: Reference to a named rule of a grammar identified by a URI alias $$aliasname#rulename <ruleref alias="aliasname#rulename"/>
2.2.3: Reference to the root rule of a grammar identified by a URI alias $$aliasname <ruleref alias="aliasname"/>
2.2.4: Special rule definitions $NULL
$VOID
$GARBAGE
<ruleref special="NULL"/>
<ruleref special="VOID"/>
<ruleref special="GARBAGE"/>

2.2.1 Local References

When referencing rules defined locally (defined in the same grammar as contains the reference), always use a simple rulename reference which consists of the local rulename only. The ABNF and XML forms have a different syntax for representing a simple rulename reference.

ABNF Form

The simple rulename reference is prefixed by a "$" character.

$city
$digit

XML Form

The "ruleref" element is an empty element with a "uri" attribute that specifies the rule reference as a fragment.

<ruleref uri="#city"/>
<ruleref uri="#digit"/>

2.2.2 External Reference by URI

References to rules defined in other grammars are legal under the conditions defined in Section 3. The external reference must identify the external grammar and may identify a specific rule within that grammar. If the rulename fragment is omitted then the reference targets the "root" rule of the external grammar.

A URI reference may optionally be accompanied by an indication of the grammar's Media type. If omitted, the grammar type is determined by other means (e.g. http header).

A URI reference is illegal if the referring document and referenced document have different modes. For instance, it is illegal to reference a "dtmf" grammar from a "voice" grammar. (See Section 4.1.3 for additional detail on modes.)

See Appendix G for a summary of the status for media types for ABNF and XML grammars.

A future draft should specify the behavior of a grammar processor when the media type in a referring document is inconsistent with the media type of the referenced grammar when determined by other means (e.g. http header or the declarations of the referenced grammar).

ABNF Form

The URL for the external grammar and optional rulename fragment are enclosed in a parentheses following a "$" symbol.

// References to specific rules of an external grammar
$(http://www.mygrammars.com/world-cities.gram#canada)
$(http://www.example.com/numbers.gram#digit)

// Reference to the root rule of an external grammar
$(../date.gram)

If a media type is specified, it is delimited by parentheses and affixed to the URI immediately following a "~" symbol (with no intervening whitespace).

Note: the media type of "application/grammar" has been requested for ABNF but is not yet granted. See Appendix G for details.

// References with associated media types
$(http://www.mygrammars.com/world-cities.gram#canada)~(application/grammar)
$(../date.gram)~(application/grammar)

Issue: the open parenthesis '(' and close parenthesis ')' symbols are both permitted with a URI. It is currently a syntax error to include the ')' error in the $(uri) construct. A grammar author may work around this limitation by using the standard percent escape syntax defined for URIs in RFC 1630. Replace the '(' and ')' symbols within the URI by '%28' and '%29' respectively. A future draft may adopt different URI delimiter characters.

XML Form

The "ruleref" element is an empty element with a "uri" attribute that optionally specifies the rule reference as a fragment.

<!-- References to specific rules of an external grammar -->
<ruleref uri="http://www.grammars.com/world-cities.xml#canada"/>
<ruleref uri="http://www.example.com/numbers.xml#digit"/>

<!-- Reference to the root rule of an external grammar -->
<ruleref uri="../date.gram"/>

The optional "type" attribute specifes the media type of the grammar containing the reference.

Note: the media type of "application/grammar+xml" has been requested for XML grammars but is not yet granted. See Appendix G for details on media types for grammars.

<!-- References with associated media types -->
<ruleref uri="http://www.grammars.com/world-cities.xml#canada" type="application/grammar+xml"/>
<ruleref uri="../date.xml" type="application/grammar+xml"/>

2.2.3 External Reference by Alias

Section 4.2 defines alias declarations. An alias defines a local name for an external grammar. The external grammar is identified by its URI. The rule reference syntax has a special mechanism to support reference to rules in grammars by the alias name.

As with reference by URI, the reference must identify the grammar name and may optionally specify the name of a rule defined within that grammar. If the rulename is omitted, the root rule of the grammar is referenced.

A alias is illegal if the referring document and referenced document have different modes. For instance, it is illegal to create an alias for a "dtmf" grammar within a "voice" grammar. (See Section 4.1.3 for additional detail on modes.)

ABNF Form

A reference by alias consists of the "$$" symbols to mark the reference followed by the alias name and optionally by the fragment separator "#" and the rulename within the referenced grammar.

// Reference a specific rule of the grammar identified by the URI alias 
$$places#city

// Reference the root rule of the grammar identified by the URI alias 
$$places

XML Form

Instead of the "uri" attribute, an alias attribute is used with a special syntax (that is intended to look somewhat like a URI with a fragment). The value of the alias attribute is the alias name followed optionally by the hash separator "#" and then the rulename within the referenced grammar.

<!-- Reference a specific rule of the grammar referenced by the alias  -->
<ruleref alias="places#city"/>


<!-- Reference the root rule of the grammar referenced by the alias  -->
<ruleref alias="places"/>

2.2.4 Special Rules

Several rulenames are defined to have specific interpretation and processing by a speech recognizer. A grammar must not redefine these rulenames.

2.2.5 Referencing N-gram Documents

The Voice Browser Working Group has released a Working Draft for the Stochastic Language Models (N-Gram) Specification in parallel with this specification. These two specifications represent different and complementary ways of informing a speech recognizer of which words and patterns of words to listen for.

A speech recognizer may choose to support the Speech Recognition N-Gram Grammar Specification in addition to the Speech Recognition Grammar Specification defined in this document.

If a speech recognizer supports both grammar representations it may optionally support references between the two formats. Grammars defined in the ABNF or XML formats may reference start symbols of N-Gram documents and vice versa.

The syntax for referencing an N-Gram is the same as referencing externally defined ABNF or XML grammar documents. Both URI references and alias referencing methods are permitted in both the ABNF and XML forms. A media type is permitted on a reference to an N-gram document. The working group has not yet applied for a type on N-gram documents so no example is given. The fragment identifier (a rulename when referencing ABNF and XML grammars) identifies a start symbol as defined by the N-Gram specification. If the start symbol is absent the N-Gram, as a whole, is referenced as defined in the N-Gram specification.

ABNF Form

URI references and alias references to N-Gram documents follow the same syntax as references to other ABNF/XML grammar documents. The following are examples of URI reference and alias reference each with an explicit rule reference and a reference to the root rule.

$(http://www.mygrammars.com/ngram.xml#StartSymbol)
$(http://www.mygrammars.com/ngram.xml)
$$ngram#StartSymbol
$$ngram

XML Form

URI references and alias references to N-Gram documents follow the same syntax as reference to other ABNF/XML grammar documents. The following are examples of URI reference and alias reference each with an explicit rule reference and a reference to the root rule.

<ruleref uri="http://www.mygrammars.com/ngram.xml#StartSymbol"/>
<ruleref uri="http://www.mygrammars.com/ngram.xml"/>
<ruleref alias="ngram#StartSymbol"/>
<ruleref alias="ngram"/>

2.3 Sequences

A sequence of legal rule expansions is itself a legal rule expansion.

The sequence of rule expansions in the grammar implies the temporal order in which the expansions must be spoken by the user and detected by the speech recognizer. This constraint applies to sequences of tokens, sequences of rule references, sequences of tags, parentheticals and all combinations of these rule expansions.

ABNF Form

A sequence of legal expansions is a white-space-separated string of the concatenated sub-expansions. Where necessary, the sequence can be delimited at the start and end by parentheses.

this is a test           // sequence of tokens
$action $object          // sequence of rule references
the $object is $color    // sequence of tokens and rule references
(fly to $city)           // parentheses for explicit boundaries
Special cases

An empty parenthetical is legal as is a parenthetical containing only whitespace; e.g. '()' or '( )'. Both forms are equivalent to $NULL and a grammar processor will behave as if the parenthetical were not present.

// equivalent sequences
phone home
phone ( ) home

XML Form

A sequence of XML rule expansion elements (<ruleref>, <item>, <one-of>, <token> <tag>) and CDATA sections containing space separated tokens must be recognized in temporal sequence. (The only exception is where one or more "item" elements appear within a "one-of" element.)

If necessary an "item" element can surround the elements of a sequence to permit a repeat attribute to be attached. The "weight" attribute of "item" is ignored unless the element appears within a "one-of" element.

<!-- sequence of tokens -->
this is a test

<!--sequence of rule references-->
<ruleref uri="#action"/> <ruleref uri="#object"/>

<!--sequence of tokens and rule references-->
the <ruleref uri="#object"/> is <ruleref uri="#color"/>

<!-- sequence container -->
<item>fly to <ruleref uri="#city"/> </item>
Special cases

An empty item element is legal as is an item element containing only whitespace. Both forms are equivalent to a NULL reference and a grammar processor will behave as if the item were not present.

// equivalent sequences
phone home
phone <item/> home
phone <item></item> home
phone <item>    </item> home

2.4 Alternatives

A set of alternative rule expansions is itself a legal rule expansion. For input to match a set of alternative rule expansions it must match one of the set of alternative expansions. A set of alternatives must contain one or more alternatives.

2.4.1 Weights

Weights may be optionally provided for each alternative expansion. Weights are simple positive floating point values ("nnn", ".nnn", "nnn.nnn") . It is legal to attach weights to alternatives in any of the following ways:

A weight is nominally a multiplying factor in the likelihood domain of a speech recognition search. The following are informative references on the topic of speech recognition technology and the underlying statistical framework within which weights are applied.

Statistical methods for speech recognition, Frederic Jelinek, ISBN: 0-262-10066-5, 1998
Fundamentals of Speech Recognition, Lawrence Rabiner and Biing-Hwang Juang, ISBN: 0-13-015157-2, 1993

It follows from the definition of weights as a multiplying factor in the likelihood domain that:

The default value for a weight if none is specified on an alternative is "1.0" (it follows from above that omitting the default weight has no impact on speech recognition).

Grammar authors and speech recognizer developers should be aware of the following limitations upon the definition and application of weights as outlined above.

Despite the limitations listed above the contributors to this document believe that reasonable interoperability of weights is achievable for the majority of grammars and speech recognizers. We define reasonable interoperability as follows.

Take grammar G that includes weights created for Recognizer A. Create grammar G' that is identical to grammar G except that all weights have been stripped. Assume that grammar G achieves better performance on Recognizer X than grammar G'. Grammar G is said to be reasonable interoperable if for the majority of comparable recognizers (similar with respect to language, supported grammar size etc.) Grammar G performs at least as well as or better than Grammar G'.

A recognizer for which tuned grammars are not reasonably interoperable should use a non-standard weight definition syntax so as not to dilute the portability of grammars implemented to this standard.

ABNF Form

A set of alternative choices is identified as a list of legal expansions separated by the vertical bar symbol. If necessary, the set of alternative choices may be delimited by parentheses.

Michael | Yuriko | Mary | Duke | $otherNames
(1 | 2 | 3)

A weight is surrounded by forward slashes and placed before each item in the alternatives list.

// Weight above 1.0 is a positive bias
// Weight below 1.0 is a negative bias
// Default is 1.0 which does not affect recognition
/10/ small | /2/ medium |  large
/3.1415/ pie | /1.414/ root beer | /.25/ cola
Special Cases

Is it legal for an alternative to be a reference to $NULL, an empty parenthetical or a single tag. In each case the input is equivalent to matching $NULL and as a result the other alternatives are optional.

// Legal
$rule1 = word | $NULL;
$rule2 = () | word;
$rule3 = word | {tag};

An empty alternative (whitespace only) is not legal.

// ILLEGAL
$rule1 = a | | b;
$rule2 = | b;
$rule3 = a |;

The following construct is interpreted as a single weighted alternative.

// Legal
$rule1 = /2/ word;
$rule2 = /2/ {tag};
$rule3 = /2/ $NULL;

XML Form

The "one-of" element identifies a set of alternative elements. Each alternative expansion is contained in a "item" element. Weights are optionally indicated by the "weight" attribute on the "item" element.

<one-of>
  <item>Michael</item>
  <item>Yuriko</item>
  <item>Mary</item>
  <item>Duke</item>
  <item><ruleref uri="#otherNames"/></item>
</one-of>

<one-of><item>1</item> <item>2</item> <item>3</item></one-of>

<one-of>
  <item weight="10">small</item>
  <item weight="2">medium</item>
  <!-- Default weight is "1.0" -->
  <item >large</item>
</one-of>

<one-of>
  <!-- Weights above 1.0 are positive bias -->
  <item weight="3.1415">pie</item>
  <item weight="1.414">root beer</item>
  <!-- Weights below 1.0 are negative bias -->
  <item weight=".25">cola</item>
</one-of>
Special cases

A one-of element containing a single item is legal and requires that input match the single item. The single item may be optionally weighted.

<one-of>
  <item>word</item>
</one-of>

<one-of>
  <item weight="2.0">word</item>
</one-of>

A one-of element containing no items is legal but is equivalent to the special VOID rule -- that is, the one-of element can never be matched by user input. There is no equivalent representation in ABNF.

Is it legal for an alternative to be a reference to NULL, an empty item or a single tag. In each case the input is equivalent to matching NULL and as a result the other alternatives are optional.

<one-of>
  <item>word</item>
  <item/>
</one-of>
<one-of>
  <item>word</item>
  <item> <ruleref special="$NULL"/> </item>
</one-of>
<one-of>
  <item>word</item>
  <item> <tag>content</tag> </item>
</one-of>

2.5 Repeats

Operators are provided that define a legal rule expansion as being another sub-expansion that is optional, that is repeated zero or more times, that is repeated one or more times, or that is repeated some range of time.

ABNF Form
Example
XML Form
Example
Behavior
<n>
<6>
repeat="n"
repeat="6"
The contained expansion is repeated exactly "n" times. "n" must be "0" or a positive integer.
<m-n>
<4-6>
repeat="m-n"
repeat="4-6"
The contained expansion is repeated between "m" and "n" times (inclusive). "m" and "n" must both be "0" or a positive integer and "m" must be less than or equal to "n".
<m->
<3->
repeat="m-"
repeat="3-"
The contained expansion is repeated "m" times or more (inclusive). "m" must be "0" or a positive integer. For example, "3-" declares that the contained expansion can occur three, four, five or more times.
<0-1>
[...]
repeat="0-1" The contained expansion is optional.
Common Repeats

As indicated in the table above, an expansion that can occur 0-1 times is optional. Because optionality is such a common form the ABNF syntax provides square brackets as a special operator for representing optionality.

A repeat of "0-" indicates that an expansion can occur zero times, once or any number of multiple times. In regular expression languages this is often represented by the Kleene star ('*') which is reserved but not used in ABNF.

A repeat of "1-" indicates that an expansion can occur once or any number of multiple times. In regular expression languages this is often represented by the positive closure ('+') which is reserved but not used in ABNF.

Although it both ABNF and XML support a grammar that permits an unbounded number of input tokens it is not the case that users will speak indefinitely. Speech recognition can perform more effectively if the author indicates a more limited range of repeat occurrences.

Repeat Probabilities

Any repeat operator may specify an optional repeat probability. The value indicates the probability of successive repetition of the repeated expansion. The value must be in the floating pointing range of "0.0" to "1.0" (inclusive). Values outside this range are an error. The floating point format is one of "n", "n.nnnn", ".nnnn" (with any number of digits after the dot).

The simplest case is an optional expansion (zero or one occurrences) with a probability of say 0.6. The grammar indicates that the chance that the expansion will be matched is 60% and that the chance that the expansion will not be present is 40%.

The following is a more complex example. If a grammar indicates that the word "x" is repeated 2-4 times with a probability of "0.8" then the grammar is indicating the following occurrence probabilities:

(Note that the total sums to 100%: 20% + 16% + 64%.)

When no maximum is specified in a range (m-) the probabilities decay exponentially.

Special Cases

An optional VOID is equivalent to NULL.

Any number of repetitions of NULL is equivalent to a single NULL.

If the number of repetitions can be only zero (e.g. <0> or <0-0>) then the expansion is never matched by input and the expansion behaves as if it were not included in the rule definition.

ABNF Form

Optional expansions are delimited by square brackets: [expansion].

The following are postfix operators: "<m-n> <m->". A postfix operator is logically attached to the proceeding expansion. Postfix operators have high precedence and so are tighly bound to the immediately preceding expansion (see Section 2.8).

The following symbols are reserved for future use in ABNF: "* + ?" and must not be used at any place in a grammar where the syntax currently permits a repeat operator.

// the token "very" is optional
[very]
very <0-1>

// the rule reference $digit can occur zero, one or many times

$digit <0->

// the rule reference $digit can occur one or more times

$digit <1->

// the rule reference $digit can occur four, five or six times
$digit <4-6>

// the rule reference $digit can occur ten or more times
$digit <10->

// Examples of the following expansion
//   "pizza"
//   "big pizza with pepperoni"
//   "very big pizza with cheese and pepperoni"
[[very] big] pizza ([with | and] $topping) <0->

Repeat probabilities are only supported in the range form. The probability is delimited by slash characters and contained within the angle brackets: "<m-n /prob/>" and "<m- /prob/>"

// the token "very" is optional and is 60% likely to occur
// and 40% likely to be absent in input
very <0-1 /0.6/>

// the rule reference $digit must occur two to four times with 80% recurrence
$digit <2-4 /.8/>

XML Form

The "item" element has a "repeat" attribute that indicates the number of times the contained expansion may be repeated. The following table defines the accepted values of the attribute.

<!-- the token "very" is optional -->

<item repeat="0-1">very</item>

<!-- the rule reference to digit can occur zero, one or many times -->

<item repeat="0-"> <ruleref uri="#digit"/> </item>

<!-- the rule reference to digit can occur one or more times -->

<item repeat="1-"> <ruleref uri="#digit"/> </item>

<!-- the rule reference to digit can occur four, five or six times -->
<item repeat="4-6"> <ruleref uri="#digit"/> </item>

<!-- the rule reference to digit can occur ten or more times -->
<item repeat="10-"> <ruleref uri="#digit"/> </item>

<!-- Examples of the following expansion -->
<!--   "pizza" -->
<!--   "big pizza with pepperoni" -->
<!--   "very big pizza with cheese and pepperoni" -->

<item repeat="0-1"> 
   <item repeat="0-1"> very </item>
   big 
</item> 
pizza
<item repeat="0-">
   <item repeat="0-1">
      <one-of>
         <item>with</item>
         <item>and</item>
      </one-of>
   </item>
   <ruleref uri="#topping"/>
</item>

The repeat-prob on the item element carries the repeat probability. Repeat probabilities are supported on any item element but are ignored if the repeat attribute is not also specified.

// the token "very" is optional and is 60% likely to occur
// and 40% likely to be absent in input
<item repeat="0-1" repeat-prob="0.6">very</item>

// the rule reference $digit must occur two to four times with 80% recurrence
<item repeat="2-4" repeat-prob=".8"> <ruleref uri="#digit"/> </item>

2.6 Tags

A tag is an arbitrary string that may be included inline with in any legal rule expansion. Tags do not affect the legal word patterns defined by the grammars or the process of recognizing speech or other input given a grammar.

Any number of tags may be included inline within a rule expansion.

Tags provide information that is typically used in post-processing of speech recognition results that match a grammar (more specifically match rule definitions and rule expansions).

The W3C Voice Browser Working Group is currently developing the "Semantic Interpretation for Speech Recognition" specification. It defines a language for grammar tags that can be used to convert spoken input to a semantic result. Examples of grammar tags in this specification will be updated to following the publication of that specification.

The tag format declaration indicates the content type of all tags in a grammar. By default the tag format is assumed to be the forthcoming W3C Semantic Interpretation for Speech Recognition tag format though a grammar processor may support other formats.

Special Cases

It is legal to use a tag as a stand-alone expansion. For example, a rule may expand to a single tag and no tokens.

  $rule = {content};
  <rule id="rule"><tag>content</tag></rule>

ABNF Form

A tag is delimited by either a pair of opening and closing curly braces -- '{' and '}' -- or by the following 3-character sequences which are considered very unlikely to occur within a tag -- '{!{' and '}!}'. The contents of the tag are not parsed by the grammar processor.

Closing braces contained within the tag may be escaped with a backslash. A backslash must also be escaped with a backslash.

Tag precedence is the same as for rule references and tokens. In the first example below there is a sequence of six space-separated expansions (3 tokens, a tag, a token and a tag). In the second example, the alternative is a choice between a sequence containing a token and a tag or a sequence containing a rule reference and a tag.

$rule1 = this is a {tag1} test {tag2};

$rule2 = open {action='open';} | $close {action='shut';};

$rule3 = {!{ a simple tag containing { and } without escaping }!};

XML Form

A "tag" element can be a direct child of the "item" and "rule" elements. The content of tag is CDATA.

<rule id="rule1">this is a <tag>tag1</tag> test <tag>tag2</tag> </rule>

<rule id="rule2">
   <one-of>
      <item> open <tag>action='open'</tag> </item>
      <item> <ruleref uri="close"/> <tag>action='shut'</tag> </item>
   </one-of>
</rule>

2.7 Language and Locale

In situations where applications target a multilingual user community it may be needed for grammars contain words in different languages. For instance, in response to a prompt such as: "Do you want to talk to Andre Prevost?" (a combination of an English sentence with a French name), the response may be either "yes" or "oui". In this case, it is possible to define two grammars, once each for the English and French responses. However, it is useful and simpler to define a grammar that contains both possibilities.

There is a related challenge for multilingual applications that deal with proper names (people, streets, companies, etc.) that may be spoken with different pronunciations or accents depending upon the language of origin and the speaking language. It is often impossible to predict the language that users will use to pronounce certain tokens. In fact, users may actually use different languages for different words in the same sentence, and in unpredictable ways. For instance, the name "Robert Jones" might be pronounced by a French-speaking user using the French pronunciation for "Robert" but an English pronunciation for "Jones", whereas a mono-lingual English speaker would use the English pronunciation for both words.

Both the ABNF and XML grammar forms support three ways of indicating language and locale. The language/locale is indicated by an RFC 1766 identifier or a space-separated sequence of identifiers.

[Note: RFC 1766 is deprecated by RFC 3066 but the changes are minor and of limited relevance to speech recognition. The grammar specification follows the XML specification and thus continues to reference RFC 1766.]

Language scoping: language declarations are scoped locally to a document and to a rule definition. In XML terminology, the language attribute is inherited down the document tree. Where a language change encompasses a reference to another grammar, the referenced rule and its containing grammar define the language of the reference expansion. The language in effect at the point of the rule reference does not have any effect upon the referenced rule.

Attaching a language identifier to grammar constructs is an indication to the speech recognizer that for the tokens contained within the construct, it should be using pronunciation rules, phonetic inventory and acoustic models corresponding to the specified language identifier. If several languages are specified for a given token, then the pronunciation, phonetic inventory and acoustic models of each language should all be used in parallel.

ABNF Form

For the ABNF form a language identifier is attached to a token or a rule expansion using the exclamation mark as delimiter. The language identifier applies to all tokens within the rule expansion. For token attachment with multiple language, the language identifiers are comma separated (cf. space-separated in XML form).

#ABNF 1.0 ISO-8859-1;

// default grammar language is US English
language en-US;

// single language attachment to tokens
$yes = oui!fr-CA | yes!en-US

// single language attachment to a rule expansion
$request = May I speak to (Michel Tremblay | André Roy)!fr-CA

// multiple language attachment to a token
$people1 =  Robert!en-US,fr-CA;

// and the equivalent single-language attachment expansion
$people2 =  Robert!en-US | Robert!fr-CA

XML Form

For the XML form the "lang-list" attribute can be attached to any of the rule expansion elements: "one-of", "token", or "item". The "lang-list" attribute may be attached to a "ruleref" element but has no effect because of the scoping rules. In addition, the "token" element can be used with the "lang-list" attribute for specifying multiple languages for an individual token by a space-separated list of locales.

<?xml version="1.0"?>

<!-- the default grammar language is US English -->
<grammar xmlns="http://www.w3.org/2001/06/grammar" xml:lang="en-US" version="1.0">

<!-- single language attachment to tokens -->
<rule id="yes">
<one-of>
 <item lang-list="fr-CA">oui</item>
  <item lang-list="en-US">yes</item>

</one-of> 
</rule> 

<!-- single language attachment to a rule expansion -->
<rule id="request">
may I speak to
<one-of lang-list="fr-CA">

  <item>Michel Tremblay</item>
  <item>André Roy</item>
</one-of>
</rule>

<!-- multiple language attachment to a token -->
<rule id="people1">
<token lang-list="en-US fr-CA"> Robert </token>

</rule>

<!-- the equivalent single-language attachment expansion -->
<rule id="people2">
<one-of>
 <item lang-list="en-US">Robert</item>
  <item lang-list="fr-CA">Robert</item>

<one-of>
</rule>

</grammar>

2.8 Precedence

This section defines the precedence of the rule expansion syntax. Because XML documents explicitly indicate structure there is no ambiguity and thus a precedence definition is not required. The precedence definitions for the ABNF form are intended to minimize the need for parentheses.

ABNF Form

The following is the ordering of precedence of rule expansions. Parentheses are used when necessary to explicitly control rule structure.

  1. Rulename denoted by the dollar sign '$', a quoted token, an unquoted token or a tag.
  2. "()" parentheses for grouping and "[]" for optional grouping.
  3. Repeat counts (e.g. '<0-1>) apply to the tightest immediate preceding rule expansion. (To apply them to a sequence or to alternatives, use `()' or `[]' grouping.)
  4. Sequence of rule expansions.
  5. `|' separated set of alternative rule expansions.

XML Form

None required. XML structure is explicit.

3. Rule Definitions

A rule definition associates a legal rule expansion with a rulename. The rule definition is also responsible for defining the scope of the rule definition: whether it is local to the grammar in which it is defined or whether it may be referenced within other grammars. Finally, the rule definition may additionally include documentation comments and other pragmatics.

The rulename must be unique within a grammar. The same rulename may be used in multiple grammars with the rulename resolution specification defining how to uniquely identify each rule definition.

3.1 Basic Rule Definition

The core purposes of a rule definition is to associate a legal rule expansion with a rulename.

A legal rulename must be a legal ID as defined by the Extensible Markup Language (XML) 1.0 specification. The ID constraint applies both to the XML and ABNF forms of the grammar.

A legal rulename cannot include the period character '.' or colon character ':' or hyphen character '-' (although all are legal characters of an XML ID). This constraints permits consistent syntax for the Semantic Interpretation for Speech Recognition specification associated closely with this grammar specification. This constraint applies to both the XML and ABNF forms of the grammar.

Rulenames are case-sensitive in both XML and ABNF grammars. Exact string comparison is used to resolved rulename references.

A legal rulename cannot be one of the Special Rules: namely "NULL", "VOID" or "GARBAGE".

ABNF Form

The rule definition consists of an optional scoping declaration (explained in the next section) followed by a legal rule name, an equals sign, a legal rule expansion and a closing semi-colon. The rule definition has one of the following legal forms:

$ruleName = ruleExpansion;
public $ruleName = ruleExpansion;
private $ruleName = ruleExpansion;

For example:

$city = Boston | "New York" | Madrid;
$command = $action $object;
Special Cases

An empty rule definition is illegal.

It is legal to define a rule that expands to empty parentheses and $NULL (equivalent forms). It is legal to define a rule that expands to a single tag.

// Legal
$rule = ();
$rule = $NULL;
$rule = {tag};

// ILLEGAL
$rule = ;

XML Form

A rule definition is represented by the "rule" element. The "id" attribute of the element indicates the name of the rule and must be unique within the grammar (this is enforced by XML). The contents of the "rule" element may be any legal rule expansion defined in Section 2. The "scope" attribute is explained in the next section.

<rule id="city">
   <one-of>
      <item>Boston</item>
      <item>"San Francisco"</item>
      <item>Madrid</item> 
   </one-of>
</rule>
<rule id="command">
   <ruleref uri="#action"/>
   <ruleref uri="#object"/>
</rule>
Special Cases

It is not legal to define an empty rule element or a rule element that contains only whitespace CDATA.

It is legal to define a rule that expands to an empty item empty or to a single rule reference to NULL.

It is legal to define a rule that expands to a single tag element.

<!-- Legal -->
<rule id="rule"><item/></rule>
<rule id="rule"><ruleref special="NULL"/></rule>
<rule id="rule"><tag>content</tag></rule>

<!-- ILLEGAL -->
<rule id="rule"/>
<rule id="rule"></rule>
<rule id="rule">   </rule>

3.2 Scoping of Rule Definitions

A rule definition may be defined as local to a grammar or may be externally referenceable and activatable. Rules with local scope are private. Rules that are externally referencable must be declared public. A rule defaults to private unless the scope is explicitly stated in the rule definition.

Public-scoped rules may be referenced in the rule definitions of other grammars (private-scoped cannot be so referenced). Section 4 explains the alias mechanism and namespace resolution that supports convenient representation of rule references. Aliases are purely a short-hand reference syntax and do not change the scoping behavior of rules.

One exception is that a rule declared as the root may be referenced externally even if it is a private rule. See Section 4.1.4 for details.

Rules with public scope may be activated for recognition. That is they may define the top-level syntax of spoken input. For instance, VoiceXML grammar activation may explicitly reference a single public rule or multiple public rules.

The intent of scoping is to allow a grammar author to separate working rules from exported rules that are intended for use elsewhere. The scoping mechanism defined here is closest to that of the Java(TM) Programming Language.

ABNF Form

A rule definition may be annotated with the keywords "public" or "private". If no scope is provided, the default is "private".

$town = Townsville | Beantown;
private $city = Boston | "New York" | Madrid;
public $command = $action $object;

XML Form

The "scope" attribute of the "rule" element defines the scope of the rule definition. Defined values are "public" and "private". If omitted, the default scope is "private".

<rule id="town">
   <one-of>
      <item>Townsville</item>
      <item>Beantown</item> 
   </one-of>
</rule>
<rule id="city" scope="private">
   <one-of>
      <item>Boston</item>
      <item>"San Francisco"</item>
      <item>Madrid</item> 
   </one-of>
</rule>
<rule id="command" scope="public">
   <ruleref uri="#action"/>
   <ruleref uri="#object"/>
</rule>

3.3 Example Phrases

It is often desirable to include examples of phrases that match rule definitions along with the definition. Zero, one or many example phrases may be provided for any rule definition. Because the examples are explicitly marked, automated tools can be used for regression testing and for generation of grammar documentation.

ABNF Form

A documentation comment is a C/C++/Java comment that starts with the sequence of characters /** and which immediately precedes the relevant rule definition. Zero or more "@example" tags may be contained at the end of the documentation comment. The tokenization of the example follows the tokenization and sequence rules defined in Section 2.

/**
 * A simple directive to execute an action.
 *
 * @example open the window
 * @example close the door
 */
public $command = $action $object;

XML Form

Any number of "example" elements may be provided as the initial content within a "rule" element. The tokenization of the example follows the tokenization and sequence rules defined in Section 2.

<rule id="command" scope="public">
    <!-- A simple directive to execute an action. -->
    <example> open the window </example>
    <example> close the door </example>
    <ruleref uri="#action"/> <ruleref uri="#object"/>
</rule>

4. Grammar Documents

A grammar document specifies a set of associated rules. All rules defined within that grammar are scoped within the grammar's namespace. Each rule defined within the grammar must have a unique name. In the XML format the grammar name is an XML ID and must be a unique ID within the complete XML document.

It is legal for a grammar to define no rules. The grammar cannot be used for processing input since it defines no patterns for matching user input.

4.1 Grammar Header

A grammar header declares the grammar version, character encoding, language/locale, mode and root though not all are required for all grammar documents. In the ABNF form this information is presented as self-identifying header followed by declarations. In the XML form the header information is encoded via the XML header and the root grammar element.

A grammar that complies to this specification must declare the version to be "1.0". Note: this version indicates the version of the specification implemented by the grammar and is not for versioning of the grammar content. (A meta declaration could be used for content versioning.)

This section presents the overall structure of the headers for ABNF and XML grammars. The sub-sections (4.1.1 - 4.1.5) define specific header properties for language/locale, mode and root. The following sections (4.2-4.5) define the other declarations of grammars (alias, lexicon, meta and comments).

ABNF Form

The ABNF header consists of the self-identifying header followed in strict order by three optional declarations: language declaration, mode declaration, root declaration and tag format declaration.

The first character of an ABNF document must be the "#" symbol (x23). It must be followed immediately by the exact string "ABNF" (x41 x42 x4d x46).

Next follows a single space character (x20) and the version number which must be "1.0" for this specification (x31 x2e x30).

Next follows an optional character encoding. Section 4.1.1 defines character encodings in more detail. If present, there must be a single space character (x20) between the version number and the character encoding.

The self-identifying header is finalized with a semi-colon (x3b) followed immediately by a newline. Newline handling follows XML conventions. The semi-colon must be the first character following the version number or the character encoding if is present.

For the remaining declarations of the ABNF header white space is not significant.

Also, for the remaining declarations of the ABNF header comments can be interspersed between but not within declarations.

The following are two examples of an ABNF header each including all declarations.

#ABNF 1.0 ISO-8859-1;

language en;
mode voice;
root $myRule;
#ABNF 1.0;

// A French Canadian grammar
language fr-CA;

// It's a speech grammar
mode voice;

// Here's the root rule
root $QuebecCities;

XML Form

The document header in XML grammars comprises the XML declaration, an optional DOCTYPE declaration and the root grammar element.

The version number of the XML declaration indicates which version of XML is being used. The version number of the grammar element indicates which version of the grammar specification is being used -- "1.0" for this specification.

The grammar element must designate the grammar namespace using the xmlns attribute. The namespace for XML grammars is defined to be http://www.w3.org/2001/06/grammar.

If present, the DOCTYPE should reference the standard DOCTYPE and identifier.

<!DOCTYPE grammar PUBLIC "-//W3C//DTD GRAMMAR 1.0//EN"
                  "http://www.w3.org/TR/speech-grammar/grammar.dtd">

The character encoding is defined on the XML declaration as defined by the XML specification. See Section 4.1.1 for detail.

The language/locale is defined by the xml:lang attribute on the grammar element. See Section 4.1.2 for details.

The grammar mode is defined on the grammar element. See Section 4.1.3 for details.

The root rule is defined on the grammar element. See Section 4.1.4 for details.

The tag format is defined on the grammar element. See Section 4.1.5 for details.

The following are two examples of XML grammars headers each including all declarations and one with the DOCTYPE declaration.

<?xml version="1.0" encoding="ISO-8859-1"?>
<grammar version="1.0" xmlns="http://www.w3.org/2001/06/grammar"
         xml:lang="en" mode="voice" root="myRule">
<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE grammar PUBLIC "-//W3C//DTD GRAMMAR 1.0//EN"
                  "http://www.w3.org/TR/WD-speech-grammar/grammar.dtd">

<grammar version="1.0" xmlns="http://www.w3.org/2001/06/grammar"
         xml:lang="fr-CA" mode="voice" root="QuebecCities">

4.1.1 Character Encoding

The character encoding indicates the symbol set used in the document. For example, for US applications it would be common to use ASCII or ISO-8859-1. . For Japanese grammars, character sets such as JIS and Unicode could be used.

Except for the different syntactic representation, the ABNF grammar format follows the character encoding behavior defined for XML. XML grammar processors must accept both the UTF-8 and UTF-16 encodings of ISO/IEC 10646. This follows from an XML grammar processor being a compliant XML processor and thus required to support those character encodings. For consistency, ABNF grammar processor must also accept both the UTF-8 and UTF-16 encodings of ISO/IEC 10646.

For both XML and ABNF grammars the character encoding is optional but recommended. XML defines behavior for XML processors that receive an XML document without a character encoding. For consistency an ABNF grammar processor must follow the same behavior (with adjustments for the different syntax).

ABNF Form

The character encoding is part of the self-identifying grammar header defined in Section 4.1.

The following are examples of ABNF self-identifying grammar headers with and without the character encoding.

#ABNF 1.0 ISO-8859-1;
#ABNF 1.0 JIS;
#ABNF 1.0;

XML Form

XML defines character encodings as part of the document's XML declaration on the first line of the document.

The following are examples of XML headers with and without the character encoding.

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml version="1.0" encoding="JIS"?>
<?xml version="1.0"?>

4.1.2 Grammar Locale

The Locale of a grammar indicates the primary language contained by the document and optionally indicates a country or other variation. Following the XML specification a locale is identified by an RFC 1766 language / country code. A language code is required by RFC 1766. A country code or other subtag identifier is optional by RFC 1766.

[Note: RFC 1766 is deprecated by RFC 3066 but the changes are minor and of limited relevance to speech recognition. The grammar specification follows the XML specification and thus continues to reference RFC 1766.]

The locale declaration is required for all speech recognition grammars: i.e. all grammars for which the mode is "voice". (Note that mode defaults to voice if there is no explicit mode declaration in ABNF or mode attribute in XML.)

If an XML grammar is incorporated within another XML document -- for example, as supported by VoiceXML 2.0 -- then the xml:lang attribute may be optional on the grammar element element and the xml:lang attribute must be inherited from the enclosing document.

The locale declaration is optional and ignored for DTMF grammars: i.e. all grammars for which the mode is "dtmf".

The Conformance definition in Section 5 defines the behavior of a grammar processor when it encounters a language variant that it does not support.

ABNF Form

If present, the language declaration is the first non-comment declaration of an ABNF grammar file following the self-identifying header.

Whitespace may appear before the "language" keyword, must appear between the keyword and the RFC 1766 code, may appear between the RFC 1766 code and the semi-colon and may appear after the semi-colon.

Comments are not permitted between the keyword and the terminating semi-colon.

language en-US;
language fr;

XML Form

Following the XML convention the language and variant are indicated by a "xml:lang" attribute on the "grammar" element. The version of the grammar is represented by the "version" attribute which is "1.0" for this specification.

<grammar xmlns="http://www.w3.org/2001/06/grammar" xml:lang="en-US" version="1.0">
<grammar xmlns="http://www.w3.org/2001/06/grammar" xml:lang="fr" version="1.0">

4.1.3 Grammar Mode

The mode of a grammar indicates the type of input that the user agent should be detecting. The default mode is "voice" for speech recognition grammars. An alternative input mode defined in Appendix E is "dtmf" input.

The mode attribute indicates how to interpret the tokens contained by the grammar. Speech tokens are expected to be detected as speech audio that sounds like the token. DTMF tokens, if supported, are detected as per the ITU Recommendation Q.24.

It is often the case that a different user agent is used for detecting DTMF tones than for speech recognition. The same may be true for other modes defined in future revisions of the specification.

The specification does not define a mechanism by which a single grammar can mix modes: that is, a representation for a mixed "voice" and "dtmf" grammar is not defined. Moreover, it is illegal for a rule reference in one grammar to reference any grammar with a different mode. Similarly, it is illegal to define a alias reference to a grammar of a different mode.

A user agent may, however, support the simultaneous activation of more than one grammar including both "voice" and "dtmf" grammars. This is necessary, for example, for VoiceXML browsers. (Parallel activation implies disjunction at the root level of the grammars rather than mixing of modes within the structure of the grammars.)

ABNF Form

An optional mode declaration should follow the language declaration if present or otherwise be the first non-comment declaration of an ABNF grammar file following the self-identifying header. If the mode declaration is omitted the default of "voice" is assumed. If the mode declaration is "dtmf" then the tokens contained by the grammar comprise DTMF events as defined in Appendix E.

mode voice;
mode dtmf;

XML Form

The mode declaration is provided as an optional "mode" attribute on the root "grammar" element. Legal values are "voice" (the default) and "dtmf".

<grammar mode="voice"
         version="1.0" xml:lang="en-US" xmlns="http://www.w3.org/2001/06/grammar">
<grammar mode="dtmf" 
         version="1.0" xml:lang="en-US" xmlns="http://www.w3.org/2001/06/grammar">

4.1.4 Root Rule Declaration

The grammar specification permits rule references to target either a specific public rule definition of an external grammar or an explicitly declared "root" rule defined within another grammar. The syntax for referencing root rules is defined in Section 2.2. It is an error to reference a grammar by its root if that grammar does not declare a legal root rule.

Both the XML and ABNF forms permit the grammar header to optionally declare a single rule to be the root rule of the grammar. A declared root rule must be a rule that is defined within the scope of the grammar.

The declared root rule may be scoped as either public or private. In the case of a public rule there is no performance difference between external references to it as either root or by its name. A private rule declared as the root may only be referenced externally as the root.

The specified rule is the rule that is referenced when a "ruleref" element references the grammar without a rulename identifier (applies both to URI references and alias references).

Although a grammar is not required to declare a root rule it is good practice to declare the root rule of any grammar.

ABNF Form

An optional root rule declaration must follow the language and mode declarations (if present) or otherwise be the first non-comment declaration of an ABNF grammar file following the self-identifying header. The root declaration must identify one rule defined elsewhere within the same grammar.

root $rulename;

XML Form

The root rulename declaration is provided as an optional "root" attribute on the "grammar" element. The root declaration must identify one rule defined elsewhere within the same grammar.

<grammar root="rulename" ...>

4.1.5 Tag Format Declaration

The tag-format declaration indicates the content type of the tags contained within a grammar. When specified, the content of all tags in the grammar must be of the declared type.

The tag format identifier is specified by the specification of the tag format. It is recommended that the tag format identifier consist of both a tag format name and a version number separated by a slash character.

The W3C Voice Browser Working Group is currently drafting the Semantic Interpretation for Speech Recognition specification. When available the W3C semantic tags will be the default tag format and all conforming grammar processors will be required to support the format. Grammar processors may support other tag formats.

The identifier for the W3C semantic tags is expected to be "semantics/1.0". This identifier is the default tag format for grammars that implement this version of the grammar specification that do not explicitly declare the tag format of a grammar.

A grammar that contains tags in a format other than W3C semantic tags must have an explicit tag format declaration specifying the format.

ABNF Form

The "tag-format" declaration, if present, must follow the language, mode and root declaration. The "tag-format" declaration, if present, must precede all alias, lexicon and meta declarations and all rule definitions.

tag-format semantics/1.0;

XML Form

The "tag-format" is an optional attribute of the grammar element.

<grammar tag-format="semantics/1.0" ...>

4.2 Aliases

An alias is a convenience mechanism for referencing externally defined grammars and public rules of those grammars. An alias declaration assigns a local name for an external grammar identified by its URI. Rule references (as defined in Section 2.2.2) can use the alias instead of the full URI when referencing rules of the aliased grammar. The intended use of aliases is to improve grammar readability when an external grammar is referenced many times.

Alias names must be unique within a grammar. As with other URI references the alias declaration does not copy the referenced grammar but simply permits reference by the short-hand name.

Alias names are not in the same namespace as rulenames. It is legal to declare both an alias and a rule definition with the same name.

The URI declared in the alias may be any absolute or relative URI but must not include a fragment separator.

As with external grammar URI references (Section 2.2.2), an optional media type specifier is permitted on an alias declaration. If omitted, the grammar type is determined by other means (e.g. http header).

The media types for the ABNF and XML Forms of this grammar specification are not yet determined.

ABNF Form

Zero, one or many "alias" declarations follow the optional "language" declaration, but preceed the rule definitions in the body of the grammar. The following alias statements define for external grammars.

alias $(http://www.mygrammars.com/cities-states.gram) $$places;



// References the "city" rule defined in the "places" grammar
... $$places#city ...

The optional media-type declaration is permitted in the same format described for external grammar rule references in Section 2.2.2.

alias $(http://www.mygrammars.com/cities-states.gram)~(application/grammar) $$places;

XML Form

Zero, one or many "alias" elements may be contained as the leading elements within a "grammar" element. The "alias" elements must preceed the "rule" elements. The "alias" element is an empty element. The "uri" and "name" attributes are required but the "type" attribute is optional.

<alias uri="http://www.mygrammars.com/cities-states.xml"
        name="places"/>


 <!-- Reference the "city" rule in the "places" grammar -->
 ... <ruleref alias="places#city"/> ...

The media-type declaration for the referenced grammar may be declared by the type attribute (same syntax as the media type declaration for an external grammar rule references as described in Section 2.2.2).

<alias uri="http://www.mygrammars.com/cities-states.xml"
        name="places" type="application/grammar+xml"/>

4.3 Pronunciation Lexicon

A grammar may optionally reference one or more external pronunciation lexicon documents. The pronunciation information contained within a lexicon document is used only for tokens defined within the enclosing grammar. If a pronunciation(s) for a token is located in the external lexicon then this pronunciation(s) is used in addition to any pronunciation(s) located in the platform default lexicon.

The format of the pronunciation lexicon document is platform specific until the specification of a standard W3C Pronunciation Lexicon Markup Language. As a consequence the matching process between the token text and the contents of the pronunciation lexicon is also platform specific.

ABNF form

Zero, one of many "lexicon" declarations may follow the optional "alias" declarations. If present, the "lexicon" declaration(s) must preceed all meta declarations and all rule definitions in the body of the grammar.

#ABNF V1.0 ISO-8859-1;
   language en-US;
   alias $(http://www.example.com/grammar.gram) $$mygrammar;
   lexicon $(http://www.example.com/lexicon.file);
   lexicon $(http://www.example.com/strange-city-names.file);
   ...

XML Form

Zero, one or many "lexicon" elements may occur as children of the "grammar" element. The "lexicon" element has a single attribute "uri" specifying the location of the pronunciation lexicon document.

<grammar xmlns="http://www.w3.org/2001/06/grammar" 
            xml:lang="en" version="1.0">
   <alias uri="http://www.example.com/places.xml" name="someplaces"/>
   <lexicon uri="http://www.example.com/lexicon.file"/>
   <lexicon uri="http://www.example.com/strange-city-names.file"/>
   ...

4.4 Meta Declarations

Grammar documents let authors specify meta data -- information about a document rather than document content -- in a variety of ways. Meta information appears in the grammar header before alias and lexicon declarations.

For example, a "meta" may be used to specify the author of a XML grammar document. In the following example, the "meta" element specifies a property (here "Author") and assigns a value to it (here "Stephanie Williams").

meta "Author" is "Stephanie Williams";
<meta name="Author" content="Stephanie Williams"/>

This document does not specify required meta-data properties, but the following are recommended (following HTML and VoiceXML):

Name Content description
author Information describing the author.
copyright A copyright notice.
description A description of the document for search engines.
keywords Keywords describing the document.
maintainer The document maintainer's email address.
robots Directives to search engine web robots.

The second type of <meta> specifies HTTP response headers. Attributes for this usage of <meta> are:

Attribute Name Description
name The name of the meta-data property.
content The value of the meta-data property.
http-equiv The name of an HTTP response header. Either name or http-equiv must be specified, not both.

This specification defines equivalent ABNF and XML representations of both forms of meta content declaration.

ABNF Form

Meta information uses either the "meta" declaration or the "http-equiv" declaration in the document header. If present, meta and http-equiv declarations must follow the self-identifying header, language, mode, root, alias and lexicon declarations (if present) and must preceed all rules in the body of the grammar. meta and http-equiv declarations may be mixed.

#ABNF 1.0;

meta "author" is "Stephanie Williams";
meta "maintainer" is "mhunt@acme.com";
...

Declarations of http-equiv use a separate declaration format but appear in the same section of the document header.

#ABNF 1.0;

http-equiv "Expires" is "0";
http-equiv "Date" is "Thu, 12 Dec 2000 23:27:21 GMT"; 
...

XML Form

Document properties declared with "meta" elements use the <meta> element with the "name" and "content" attributes. One or more <meta> elements may optionally follow the document's grammar element as shown in the following example.

<?xml version="1.0"?> 
<grammar version="1.0" xml:lang="en-US"
        xmlns="http://www.w3.org/2001/06/grammar"> 
  <meta name="author" content="Stephanie Williams"/> 
  <meta name="maintainer" content="mhunt@acme.com"/> 
   ... 
</grammar>

In the following example, the first <meta> element sets an expiration date that prevents caching of the document; the second <meta> element sets the Date header.

<?xml version="1.0"?> 
<grammar version="1.0" xml:lang="en-US"
         xmlns="http://www.w3.org/2001/06/grammar"> 
  <meta http-equiv="Expires" content="0"/> 
  <meta http-equiv="Date"    content="Thu, 12 Dec 2000 23:27:21 GMT"/> 
   ... 
</grammar>

4.5 Comments

Comments may be placed in most places in a grammar document. For XML, use XML comments. For ABNF there are documentation comments and C/C++/Java-style comments.

ABNF Form

C/C++/Java comments are permitted. Documentation comments are permitted before grammar, language and alias declarations and before each rule definition.

Section 3.3 defines the format for representing examples in documentation comments before a rule definition.

// C++/Java-style single-line comment
/* C/C++/Java-style comment */
/** Java-style documentation comment */

XML Form

An XML comment has the following syntax.

<!-- comment -->

4.6 Grammar Fetching

The fetching and caching behavior of both ABNF and XML grammar documents is defined primarily by the environment in which the grammar processor operates. For instance, VoiceXML 1.0 and VoiceXML 2.0 define certain fetching and caching behaviors that apply to grammars activated by a VoiceXML browser. Similarly, any API for a recognizer that supports ABNF or XML grammars may apply fetching and caching behaviors.

Grammar processors are recommended to support the following interpretation of "rendering" a grammar for the purpose of determine document freshness.

Activation of a grammar is the point at which the recognizer begins detection of user input matching the grammar and is therefore analogous to the action of visual or audio rendering of system output. As with output rendering, grammar freshness should be checked close to the moment of grammar activation.

4.7 ABNF Keywords

The keywords of the ABNF language are not reserved. The keywords with specified meaning in ABNF are:

Context Keywords
Language declaration "language"
Mode declaration "mode"
Root declaration "root"
Alias declaration "alias"
Pronunciation lexicon "lexicon"
Meta or HTTP-equiv declaration "meta", "http-equiv", "is"
Rule definition "public", "private"

Since keywords are not reserved they may be used as alias names, rulenames and as tokens. The following is a legal grammar (albeit extreme and useless).

#ABNF 1.0 ISO-8859-1;

language en-AU;
root $public;
mode voice;

alias $(http://public.example.com/public.gram) $$public;

public $public = public $public | $$public;

5. Conformance

This section is Normative.

Different sets of grammar conformance criteria exist for:

5.1: Conforming XML Grammar Fragments

An XML grammar document fragment is a Conforming XML Document Fragment if it adheres to the specification described in this document (Speech Recognition Grammar Format Specification) including the DTD (see Appendix B) and the Schema (see Appendix C) and also:

The XML grammar language or these conformance criteria provide no designated size limits on any aspect of grammar documents. There are no maximum values on the number of elements, the amount of character data, or the number of characters in attribute values.

5.2: Conforming Stand-Alone XML Grammar Document

A file is a Conforming Stand-Alone XML Grammar Document if:

5.3: Using XML Grammars with other Namespaces

The grammar namespace may be used with other XML namespaces as per the Namespaces in XML Recommendation. Future work by W3C will address ways to specify conformance for documents involving multiple namespaces.

5.4: Conforming XML Grammar Processors

An XML Grammar processor is a program that can parse and process XML Grammar fragments. Examples include speech recognizers and DTMF detectors that accept the XML Grammar format.

In a Conforming XML Grammar Processor, the XML parser must be able to parse and process all XML constructs defined within XML 1.0 and XML Namespaces.

A Conforming XML Grammar Processor must correctly understand and apply the semantics of each possible grammar feature defined by this document.

A Conforming XML Grammar Processor must meet the following requirements for handling of languages and locales:

When a Conforming XML Grammar Processor encounters elements or attributes in a non-grammar namespace it may:

A Conforming XML Grammar Processor is not required to support recursive grammars, that is, grammars in which rule references include direct or indirect self-reference.

There is, however, no conformance requirement with respect to performance characteristics of the XML Grammar Processor. For instance, no statement is required regarding the accuracy, speed or other characteristics of a speech recognizer or DTMF detector. No statement is made regarding the size of grammar or size of grammar vocabulary that an XML Grammar Processor must support.

5.5: Conforming Stand-Alone ABNF Grammar Documents

An ABNF grammar document is a Conforming ABNF Document if it adheres to the specification described in this document (Speech Recognition Grammar Format Specification) including the Formal ABNF Specification.

5.6: Conforming ABNF Grammar Processor

An ABNF Grammar processor is a program that can parse and process ABNF Grammar documents. Examples include speech recognizers and DTMF detectors that accept the ABNF Grammar format.

A Conforming ABNF Grammar Processor must correctly understand and apply the semantics of each possible grammar feature defined by this document.

A Conforming ABNF Grammar Processor must follow the same language/locale handling requirements as outlined in Section 5.4 for Conforming XML Grammar Processors.

A Conforming ABNF Grammar Processor should inform its hosting environment if it encounters an illegal grammar document or other grammar content that it is unable to process.

A Conforming ABNF Grammar Processor is not required to support recursive grammars, that is, grammars in which rule references include direct or indirect self-reference.

There is, however, no conformance requirement with respect to performance characteristics of the ABNF Grammar Processor. For instance, no statement is required regarding the accuracy, speed or other characteristics of a speech recognizer or DTMF detector. No statement is made regarding the size of grammar or size of grammar vocabulary that an ABNF Grammar Processor must support.

5.7: Conforming ABNF/XML Grammar Processor

A Conforming ABNF/XML Grammar Processor must meet all the conformance criteria defined in Section 5.4 and in Section 5.6.

Additionally an ABNF/XML Grammar Processor must be able to resolve and apply references from XML Grammars to ABNF Grammars, and references from ABNF Grammars to XML Grammars.

5.8: Conforming User Agent

A conforming user agent is a Conforming XML Grammar Processor, Conforming ABNF Grammar Processor or Conforming ABNF/XML Grammar Processor that is capable of accepting user input of the mode of a grammar (i.e. speech input in "voice" mode, DTMF input "dtmf" mode) and:

  1. Is capable of determining when a sequence of user input exactly matches a grammar,
  2. Is capable of producing an output representation that indicates how the input matches the grammar,

Issue: A future draft will require a Conforming User Agent to also support the Semantic Interpretation for Speech Recognition specification. The Semantic Interpretation specification is still in development by the W3C Voice Browser Working Group.

Current speech recognition technology is statistically based. Since the output is not deterministic and cannot be guaranteed to be a correct representation of the input there is no conformance requirement regarding accuracy. A conformance test may, however, require some examples of correct recognition of speech input to determine conformance.

6. Acknowledgements

This document was written with the participation of the members of the W3C Voice Browser Working Group (listed in alphabetical order):

Mike Brown, Lucent Bell Labs
Dan Burnett, Nuance Communications
Debbie Dahl, Unisys
Andrew Hunt, SpeechWorks International
Stefan Krause, Philips
Bruce Lucas, IBM
Jens Marschner, Philips
Scott McGlashan, PipeBeach
Yves Normandin, Locus Dialogue
Dave Raggett, W3C/OpenWave
David Ramsthaler, Cisco
Luc Van Tichelen, Lernout & Hauspie
Kuansan Wang, Microsoft

Appendix A: Example Grammars in ABNF and XML Forms

This appendix is Informative.

The following shows a simple grammar that supports commands such as "open a file" and "please move the window". It references a separately-defined grammar for politeness which is not shown here.

ABNF Form

#ABNF 1.0 ISO-8859-1;

language en;mode voice;
root $basicCmd;
alias $(http://www.sayplease.com/politeness.gram) $$polite;


meta "author" is "Stephanie Williams";

/**
 * Basic command.
 * @example please move the window
 * @example open a file
 */

public $basicCmd = 
          $$polite#startPolite $command $$polite#endPolite;

$command = $action $object;
$action = /10/ open {'OPEN'} | /2/ close {'CLOSE'} 
                 | /1/ delete {'DELETE'} | /1/ move {'MOVE'};
$object = [the | a] (window | file | menu);

XML Form

<?xml version="1.0"?>

<grammar xmlns="http://www.w3.org/2001/06/grammar" xml:lang="en" version="1.0" mode="voice" root="basicCmd">

<alias name="polite"
  uri="http://www.sayplease.com/politeness.xml"/>


<meta name="author" content="Stephanie Williams"/>

<rule id="basicCmd" scope="public">
  <example> please move the window </example>
  <example> open a file </example>

 <ruleref alias="polite#startPolite"/>

  <ruleref uri="#command"/>
 <ruleref alias="polite#endPolite"/>

</rule>

<rule id="command">
  <ruleref uri="#action"/> <ruleref uri="#object"/>
</rule>

<rule id="action">
   <one-of>
    <item weight="10"> open   <tag>'OPEN'</tag> </item>
      <item weight="2">  close  <tag>'CLOSE'</tag> </item>
      <item weight="1">  delete <tag>'DELETE'</tag> </item>
      <item weight="1">  move   <tag>'MOVE'</tag> </item>

    </one-of>
</rule>

<rule id="object">
 <item repeat="0-1">

    <one-of>
      <item> the </item>
      <item> a </item>
    </one-of>
 </item>

  <one-of>
      <item> window </item>
      <item> file </item>
      <item> menu </item>
  </one-of>
</rule>

</grammar>

The next two grammars illustrate referencing between grammars in both XML and ABNF formats.

ABNF: http://www.example.com/places.gram

#ABNF 1.0 ISO-8859-1;

language en;mode voice;
root $city_state;
// No aliases  in this referenced grammar.

public $city = Boston | Philadelphia | Fargo;

public $state = Florida | North Dakota | New York;

// References to local rules
// Artificial example allows "Boston, Florida!"

public $city_state = $city $state;

ABNF: http://www.example.com/booking.gram

#ABNF 1.0 ISO-8859-1;

language en;
mode voice;

alias $(http://www.example.com/places.gram) $$someplaces;


// Reference by URI syntax
$flight = I want to fly to
   $(http://www.example.com/places.gram#city);

// Reference using alias name
$exercise = I want to walk to $$someplaces#state;

// Reference to root rule using an alias reference
$wet = I want to swim to $$someplaces;

XML Grammar: http://www.example.com/places.xml

<?xml version="1.0"?>

<grammar xmlns="http://www.w3.org/2001/06/grammar" xml:lang="en" version="1.0" root="city_state" mode="voice">

   <rule id="city" scope="public">
     <one-of>
       <item>Boston</item>
       <item>Philadelphia</item>
       <item>Fargo</item>
     </one-of>
   </rule>

   <rule id="state" scope="public">
     <one-of>
       <item>Florida</item>
       <item>North Dakota</item>
       <item>New York</item>
     </one-of>
   </rule>

   <!-- Reference by URI to a local rule -->
   <!-- Artificial example allows "Boston, Florida"! -->
   <rule id="city_state" scope="public">
     <ruleref uri="#city"/> <ruleref uri="#state"/>
   </rule>
</grammar>

XML Grammar: http://www.example.com/booking.xml

<?xml version="1.0"?>

<grammar xmlns="http://www.w3.org/2001/06/grammar" xml:lang="en" version="1.0" mode="voice">

 <alias name="someplaces"
    uri="http://www.example.com/places.xml"/>


   <!-- Using URI syntax -->
   <rule id="flight">
     I want to fly to 
     <ruleref uri="http://www.example.com/places.xml#city"/>
   </rule>

   <!-- Using alias syntax -->
   <rule id="exercise">
   I want to walk to <ruleref alias="someplaces#state"/>

   </rule>

   <!-- Reference to root rule of a grammar by alias -->
   <rule id="wet">
   I want to swim to <ruleref alias="someplaces"/>

   </rule>
</grammar>

Appendix B: DTD for the XML Format

This appendix is Normative.

<?xml version="1.0" encoding="ISO-8859-1"?>

<!-- Speech Recognition Grammar Format DTD v0.93 20010816 -->

<!ENTITY % rule-expansion "#PCDATA | token | ruleref
                              | item | one-of | tag " >

<!ELEMENT ruleref EMPTY>
<!ATTLIST ruleref
     uri CDATA #IMPLIED
     alias CDATA #IMPLIED
     type CDATA #IMPLIED
     special (NULL | VOID | GARBAGE) #IMPLIED
     lang-list NMTOKENS #IMPLIED>

<!ELEMENT token (#PCDATA)>
<!ATTLIST token
     lang-list NMTOKENS #IMPLIED>

<!ELEMENT tag (#PCDATA)>

<!ELEMENT one-of (item)*>
<!ATTLIST one-of
     lang-list NMTOKENS #IMPLIED>

<!ELEMENT item (%rule-expansion;)*>
<!ATTLIST item
    repeat NMTOKEN #IMPLIED
    repeat-prob NMTOKEN #IMPLIED
    weight NMTOKEN #IMPLIED>

<!ELEMENT rule (%rule-expansion; | example )*>
<!ATTLIST rule 
    id ID #REQUIRED
    scope (private | public) "private">

<!ELEMENT example (#PCDATA)>

<!ELEMENT alias EMPTY>
<!ATTLIST alias
    uri CDATA #REQUIRED
    type CDATA #IMPLIED
    name NMTOKEN #REQUIRED>

<!ELEMENT lexicon EMPTY>
<!ATTLIST lexicon
    uri CDATA #REQUIRED>

<!ELEMENT meta EMPTY>
<!ATTLIST meta
    name        NMTOKEN     #IMPLIED
    content     CDATA       #REQUIRED 
    http-equiv  NMTOKEN     #IMPLIED>

<!ELEMENT grammar (alias | lexicon | meta | rule)*>
<!ATTLIST grammar 
    tag-format CDATA #IMPLIED
    version NMTOKEN #REQUIRED
    xml:lang NMTOKEN #IMPLIED
    root IDREF #IMPLIED
    mode (voice | dtmf) "voice">

Appendix C: Schema for the XML Grammar Format

This appendix is Normative.

<?xml version="1.0" encoding="ISO-8859-1"?>
<schema targetNamespace="http://www.w3.org/2001/06/grammar"
           xmlns:xml="http://www.w3.org/XML/1998/namespace"
           xmlns:t="http://www.w3.org/2001/06/grammar"
           xmlns="http://www.w3.org/2001/XMLSchema"
           elementFormDefault="qualified">
  <import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="http://www.w3.org/2001/xml.xsd"/>
  <simpleType name="lang-list">
    <restriction base="NMTOKENS"/>
  </simpleType>
  <simpleType name="tag">
    <restriction base="string"/>
  </simpleType>
  <simpleType name="example">
    <restriction base="string"/>
  </simpleType>
  <group name="rule-expansion">
    <choice>
      <element name="token" type="t:token"/>
      <element name="ruleref" type="t:ruleref"/>
      <element name="item" type="t:item"/>
      <element name="one-of" type="t:one-of"/>
      <element name="tag" type="t:tag"/>
    </choice>
  </group>
  <complexType name="ruleref">
    <attribute name="type" type="string"/>
    <attribute name="uri" type="anyURI"/>
    <attribute name="alias" type="string"/>
    <attribute name="special">
      <simpleType>
        <restriction base="NMTOKEN">
          <enumeration value="NULL"/>
          <enumeration value="VOID"/>
          <enumeration value="GARBAGE"/>
        </restriction>
      </simpleType>
    </attribute>
    <attribute name="lang-list" type="t:lang-list"/>
  </complexType>
  <complexType name="token" mixed="true">
    <annotation>
      <documentation>
does not expression the constraint that empty content is illegal
</documentation>
    </annotation>
    <attribute name="lang-list" type="t:lang-list"/>
  </complexType>
  <complexType name="one-of">
    <sequence minOccurs="0" maxOccurs="unbounded">
      <element name="item" type="t:item"/>
    </sequence>
    <attribute name="lang-list" type="t:lang-list"/>
  </complexType>
  <complexType name="item" mixed="true">
    <choice minOccurs="0" maxOccurs="unbounded">
      <group ref="t:rule-expansion"/>
    </choice>
    <attribute name="repeat-prob">
      <simpleType>
        <restriction base="float">
          <minInclusive value="0.0"/>
          <maxInclusive value="1.0"/>
          <pattern value="[0-9]+(.[0-9]+)?"/>
          <pattern value="([0-9]+)?.[0-9]+"/>
        </restriction>
      </simpleType>
    </attribute>
    <attribute name="repeat">
      <simpleType>
        <annotation>
          <documentation>
does not expression the constraint in n-m that m must be greater than n
</documentation>
        </annotation>
        <restriction base="string">
          <pattern value="[0-9]+"/>
          <pattern value="[0-9]+-([0-9]+)?"/>
          <pattern value="([0-9]+)?-[0-9]+"/>
        </restriction>
      </simpleType>
    </attribute>
    <attribute name="weight">
      <simpleType>
        <restriction base="float">
          <minInclusive value="0.0"/>
          <pattern value="[0-9]+"/>
          <pattern value="([0-9]+)?.[0-9]+"/>
        </restriction>
      </simpleType>
    </attribute>
  </complexType>
  <complexType name="rule" mixed="true">
    <choice minOccurs="0" maxOccurs="unbounded">
      <group ref="t:rule-expansion"/>
      <element name="example" type="t:example"/>
    </choice>
    <attribute name="id" use="required">
      <simpleType>
        <annotation>
          <documentation>
does not expression the constraint that NULL VOID GARBAGE are illegal as rule name
</documentation>
        </annotation>
        <restriction base="ID">
          <pattern value="[^.:-]+"/>
        </restriction>
      </simpleType>
    </attribute>
    <attribute name="scope" default="private">
      <simpleType>
        <restriction base="NMTOKEN">
          <enumeration value="private"/>
          <enumeration value="public"/>
        </restriction>
      </simpleType>
    </attribute>
  </complexType>
  <complexType name="alias">
    <attribute name="type" type="string"/>
    <attribute name="uri" type="anyURI" use="required"/>
    <attribute name="name" type="NMTOKEN" use="required"/>
  </complexType>
  <complexType name="lexicon">
    <attribute name="uri" type="anyURI" use="required"/>
  </complexType>
  <complexType name="meta">
    <attribute name="name" type="NMTOKEN"/>
    <attribute name="content" type="string" use="required"/>
    <attribute name="http-equiv" type="NMTOKEN"/>
  </complexType>
  <complexType name="grammar">
    <choice minOccurs="0" maxOccurs="unbounded">
      <element name="alias" type="t:alias"/>
      <element name="lexicon" type="t:lexicon"/>
      <element name="meta" type="t:meta"/>
      <element name="rule" type="t:rule"/>
    </choice>
    <attribute name="tag-format" type="string"/>
    <attribute name="version" type="NMTOKEN" use="required"/>
    <attribute ref="xml:lang"/>
    <attribute name="root">
      <simpleType>
        <annotation>
          <documentation>
does not expression the constraint that NULL VOID GARBAGE are illegal as rule name
</documentation>
        </annotation>
        <restriction base="IDREF">
          <pattern value="[^.:-]+"/>
        </restriction>
      </simpleType>
    </attribute>
    <attribute name="mode" default="voice">
      <simpleType>
        <restriction base="NMTOKEN">
          <enumeration value="voice"/>
          <enumeration value="dtmf"/>
        </restriction>
      </simpleType>
    </attribute>
  </complexType>
  <element name="grammar" type="t:grammar"/>
</schema>

Appendix D: Formal Syntax for Augmented BNF

This appendix is Normative.

The notation used here follows the EBNF notation defined by the XML 1.0 specification.

Lexical Grammar for ABNF

SelfIdentHeader  ::= '#ABNF' #x20 VersionNumber (#x20 CharEncoding)? ';'

VersionNumber    ::= Nmtoken                           
     
CharEncoding     ::= Nmtoken

LanguageCode     ::= Nmtoken

RuleName         ::= '$' ConstrainedName

TagFormat        ::= (NameChar | '/')+

GrammarURI       ::= '$' '(' URIC+ ')'                 /* in the DTD: CDATA */

MimeType         ::= '(' MimeTypeChar+ ')'

LexiconURI       ::= '(' URIC+ ')'                     /* in the DTD: CDATA */

AliasName        ::= '$$' ConstrainedName              

QuotedCharacters ::= '"' CharData '"'
     

Weight           ::= '/' [0-9]+ '/' | '/' [0-9]* '.' [0-9]+ '/'

RepeatOperator   ::= '+' | '*' | '<' [0-9]+ ('-' [0-9]*)? (' ' Probability)? '>'

Probability      ::= '1' | '0' | '1' '.' '0'+ | '0'? '.' [0-9]+

URIRuleRef       ::= '$' '(' URIC+ '#' Name ')'

URIRootRef       ::= '$' '(' URIC+ ')'

AliasRuleRef     ::= AliasName '#' ConstrainedName

AliasRootRef     ::= AliasName

Token            ::= Nmtoken | QuotedCharacters

----------------------------------------------------------------------------------

URIC is defined by the uric production of RFC2396.
It is the set of all characters, which are allowed within a URI.

Name is defined by the XML Name production.

ConstrainedName is a Name, which does not contain a dot, colon or hyphen.

Nmtoken is defined by the XML Nmtoken production.

NameChar is defined by the XML NameChar production. It defines the set of
characters, which are allowed in an NMTOKEN.

CharData is defined by the XML CharData production.

Syntactic Grammar for ABNF

grammar         ::= header  declarations ruleDefinition*

header          ::= SelfIdentHeader localeDecl? modeDecl? rootDecl? tagFormatDecl?

localeDecl      ::= 'language' LanguageCode ';'

modeDecl        ::= 'mode' 'voice' ';' | 'mode' 'dtmf' ';'

rootDecl        ::= 'root' RuleName ';'

tagFormatDecl   ::= 'tag-format' TagFormat ';'


declarations    ::= aliasDecl* lexiconDecl* metaDecl*

aliasDecl       ::= 'alias' GrammarURI ('~' MimeType)?  AliasName ';'

lexiconDecl     ::= 'lexicon' LexiconURI ';'

metaDecl        ::= 'meta' QuotedCharacters 'is' QuotedCharacters ';'
                    | 'http-equiv' QuotedCharacters 'is' QuotedCharacters ';'

ruleDefinition  ::=  scope? RuleName '=' ruleExpansion ';'   [VC: unique rule name]

scope           ::= 'private' | 'public'

ruleExpansion   ::= ruleAlternative ( '|' ruleAlternative )*

ruleAlternative ::= Weight? sequenceElement+

sequenceElement ::= subexpansion | subexpansion RepeatOperator

subexpansion    ::= Token
                    | ruleRef
                    | tag
                    | '(' ')'
                    | '(' ruleExpansion ')'
                    | '[' ruleExpansion ']'
                    | Token '!' LanguageCode+
                    | '(' ruleExpansion ')' '!' LanguageCode
                    | '[' ruleExpansion ']' '!' LanguageCode

ruleRef         ::= localRuleRef | externalRuleRef | specialRuleRef

localRuleRef    ::= RuleName

externalRuleRef ::= URIRuleRef | URIRootRef | AliasRuleRef | AliasRootRef

specialRuleRef  ::= '$NULL' | '$VOID' | '$GARBAGE'

tag             ::= '{' tagContent '}' | '{!{' tagContent '}!}'

----------------------------------------------------------------------------------

tagContent will be defined by the Semantic Interpretation 
for Speech Recognition specification.

Appendix E: DTMF Grammars

This appendix is Normative.

This section defines a Normative representation of a grammar consisting of DTMF tokens. A DTMF grammar can be used by a DTMF detector to determine sequences of legal and illegal DTMF events. All grammar processors that support grammars of mode "dtmf" must implement this Appendix. However, not all grammar processors are required to support DTMF input.

DTMF (Dual Tone Multiple Frequency) is an ITU standard for telephony signaling. ITU Recommendation Q.23 defines DTMF generation. ITU Recommendation Q.24 defines DTMF reception.

If the grammar mode is declared as "dtmf" then tokens contained by the grammar are treated as DTMF tones (rather as the default of speech tokens).

There are sixteen (16) DTMF tones. Of these twelve (12) are commonally found on telephone sets as the digits "0" through "9" plus "*" (star) and "#" (pound). The four DTMF tones not typically present on telephones are "A", "B", "C", "D".

Each of the DTMF symbols is a legal DTMF token in a DTMF grammar. Space-separated DTMF symbols represent temporal sequences of DTMF entry.

In the ABNF form the "*" symbol is reserved so quotes must always be used to delimit "*" when defining an ABNF DTMF grammar. It is recommended that the "#" symbol also be quoted. As an alternative the tokens "star" and "pound" are acceptable synonyms.

If present, a language declaration in an ABNF grammar or an xml:lang attribute or lang-list attribute in an XML grammar is ignored.

In all other respects a DTMF grammar is syntactically the same as a speech grammar, including the semantic results processing.

The following is a simple DTMF grammar that accepts a 4-digit PIN followed by a pound terminator. It also permits "*9" (e.g. to receive a help message).

#ABNF 1.0 ISO-8859-1;

mode dtmf;

$digit = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9;
public $pin = $digit <4> "#" | "*" 9;
<?xml version="1.0"?>

<grammar mode="dtmf" version="1.0" xmlns="http://www.w3.org/2001/06/grammar">

<rule id="digit">
 <one-of>
   <item> 0 </item>
   <item> 1 </item>
   <item> 2 </item>
   <item> 3 </item>
   <item> 4 </item>
   <item> 5 </item>
   <item> 6 </item>
   <item> 7 </item>
   <item> 8 </item>
   <item> 9 </item>
 </one-of>
</rule>

<rule id="pin" scope="public">
 <one-of>
   <item>
     <item repeat="4"><ruleref uri="#digit"/></item>
     #
   </item>
   <item>
     * 9
   </item>
 </one-of>
</rule>

</grammar>

Appendix F: XSLT Style Sheet to Convert XML Grammars to ABNF

This appendix is Informative.

The transformation provided below is illustrative of the conversion of an XML Form grammar to the Augmented BNF Form. Known limitations:

<?xml version="1.0"?> 

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:strip-space elements= "alias rule example item one-of token"/>

<xsl:output method="text"/>

<xsl:template name="addweight">
  <xsl:if test="string(@weight)!=''">
    /<xsl:value-of select="@weight"/>/
  </xsl:if>
</xsl:template>


<xsl:template name="addlanglist">
  <xsl:if test="string(@lang-list)!=''">
    !<xsl:value-of select="translate(normalize-space(@lang-list),' ',',')"/>
  </xsl:if>
</xsl:template>


<xsl:template name="addrepeat">
  <xsl:if test="string(@repeat)!=''">
  <xsl:choose>
    <xsl:when test="string(@repeat-prob)!=''">
    &lt;<xsl:value-of select="@repeat"/> /<xsl:value-of select="@repeat-prob"/>/&gt;
    </xsl:when>
    <xsl:otherwise>
    &lt;<xsl:value-of select="@repeat"/>&gt;
    </xsl:otherwise>
    </xsl:choose>
  </xsl:if>
</xsl:template>


<xsl:template match="grammar">
  #ABNF  <xsl:value-of select="@version"/> <xsl:value-of select="system-property('xsl:encoding')"/>;
  <xsl:text> </xsl:text>
  <xsl:if test="string(@xml:lang)!=''">
  language <xsl:value-of select="@xml:lang"/>;
  </xsl:if>
  <xsl:if test="string(@mode)!=''">
  mode <xsl:value-of select="@mode"/>;
  </xsl:if>
  <xsl:if test="string(@root)!=''">
  root <xsl:value-of select="@root"/>;
  </xsl:if>
  <xsl:if test="string(@tag-format)!=''">
  tag-format <xsl:value-of select="@tag-format"/>;
  </xsl:if>
  <xsl:apply-templates select="alias"/>
  <xsl:apply-templates select="lexicon"/>
 <xsl:apply-templates select="meta"/>
  <xsl:apply-templates select="rule"/>
</xsl:template>


<xsl:template match="tag">
    {<xsl:value-of select="."/>}
</xsl:template>

<xsl:template match="alias">
  <xsl:choose>
    <xsl:when test="string(@type)!=''">
  alias $(<xsl:value-of select="@uri"/>)~(<xsl:value-of select="@type"/>) $$<xsl:value-of select="@name"/>;
    </xsl:when>
    <xsl:otherwise>
  alias $(<xsl:value-of select="@uri"/>) $$<xsl:value-of select="@name"/>;
    </xsl:otherwise>
    </xsl:choose>
</xsl:template>

<xsl:template match="lexicon">
lexicon $(<xsl:value-of select="@uri"/>);
</xsl:template>

<xsl:template match="meta">
<xsl:if test="string(@content)!=''">
<xsl:choose>
<xsl:when test="string(@http-equiv)!=''">
http-equiv "<xsl:value-of select="@http-equiv"/>" is "<xsl:value-of select="@content"/>";
</xsl:when>
<xsl:when test="string(@name)!=''">
meta "<xsl:value-of select="@name"/>" is "<xsl:value-of select="@content"/>";
</xsl:when>
</xsl:choose>
</xsl:if>
</xsl:template>

<xsl:template match="rule">
  <xsl:value-of select="@scope"/>
  $<xsl:value-of select="@id"/> = 
  <xsl:apply-templates/>
  ;
</xsl:template>

<xsl:template match="token">
  (<xsl:value-of select="text()"/>)
  <xsl:call-template name="addlanglist"/>
</xsl:template>

<xsl:template match="ruleref">
  <xsl:choose>
    <xsl:when test="string(@special)!=''">
      $<xsl:value-of select="@special"/> 
    </xsl:when>
    <xsl:when test="string(@alias)!=''">
      $$<xsl:value-of select="@alias"/>
    </xsl:when>
    <xsl:otherwise>
      <xsl:choose>
        <xsl:when test="starts-with(string(@uri),'#')">
          $<xsl:value-of select="substring-after(@uri,'#')"/>
        </xsl:when>
        <xsl:otherwise>
    <xsl:choose>
        <xsl:when test="string(@type)!=''">
          $(<xsl:value-of select="@uri"/>)~(<xsl:value-of select="@type"/>)
        </xsl:when>
    <xsl:otherwise>
          $(<xsl:value-of select="@uri"/>)
    </xsl:otherwise>
    </xsl:choose>
        </xsl:otherwise>
      </xsl:choose>
    </xsl:otherwise>
  </xsl:choose>
  <xsl:call-template name="addlanglist"/>
</xsl:template>

<xsl:template match="example"/>

<xsl:template match="one-of">  <xsl:choose>
    <xsl:when test="count(item)=0">
    $VOID
    </xsl:when>
    <xsl:otherwise>
        (<xsl:apply-templates/>)
        <xsl:call-template name="addlanglist"/>
    </xsl:otherwise>
  </xsl:choose>
</xsl:template>

<xsl:template match="item">
  <xsl:call-template name="addweight"/>
  <xsl:apply-templates/> 
  <xsl:call-template name="addrepeat"/>
  <xsl:if test="not(position()=last())">|</xsl:if>
</xsl:template>

</xsl:stylesheet>

Appendix G. Media Types and File Suffix

This appendix is Informative.

The W3C Voice Browser Working Group has applied to IETF to register media types for both the ABNF and XML grammar forms.

The ABNF Media type will identify ABNF grammars containing only semantic tags defined by the (to-be-released) Semantic Interpretation for Speech Recognition specification or no tags. If the grammar contains tags of any other format then a different media type must be used.

Similarly, the XML grammar media type will identify XML grammars containing only semantic tags defined by the (to-be-released) Semantic Interpretation for Speech Recognition specification or no tags. If the grammar contains tags of any other format then a different media type must be used.

The W3C Voice Browser Working Group has adopted the convention of using the ".gram" filename suffix for ABNF grammar documents and the ".grxml" filename suffix for XML grammar documents.

Appendix H. Logical Parse Structure

This appendix is Informative.

The W3C Voice Browser Working Group is currently developing the Semantic Interpretation for Speech Recognition specification for writing semantic tags. The execution of these tags will produce a semantic result that can be made available to a VoiceXML application. The semantic interpretation draft makes use of the logical parse structure to define order of execution and visibility rules.

H.1 Terminology and Notation

This Appendix adopts the terminology and nomenclature of Introduction to Automata Theory, Languages, and Computation, J.E. Hopcroft and J.D. Ullman, Addison-Wesley, 1979 [HU79].

Denote the tokens of the alphabet of all tokens accepted by a grammar as t1, t2....

An input or output token sequence is a space separated string of tokens. e.g. "t3 t2 t1".

Let ε (epsilon) or "" denote the unique string of length 0, also known as the empty string.

Denote the tags of the alphabet of all tags accepted by a grammar as {tag1}, {tag2}, ....

Denote a legal expansion as E. (A legal expansion is defined in Section 2.)

The expressive power of a rule expansion is a Regular Expression (see HU79) and has an equivalent Finite Automaton (see HU79). [The handling of rule references requires special treatment: see Section H.2.] The expressive power of the grammar specification consists of:

We formalize the logical parse structure by creating a Finite Automaton with Output (see HU79). This construct is also referred to as a Finite State Transducer.

We define the transitions for tokens and tags as producing an output symbol.

We represent parse output as an ordered array of output entities: [e1,e2,e3,...].

An entity e may be a token, a tag or a rule expansion (see H.2).

The empty output array is represented as [ε] or simply [].

Special Cases

A $NULL reference is equivalent to a transition that accepts as input ε and produces as output ε. In the notation of HU79: ε/ε.

A $VOID reference is logically equivalent to a missing transition. It accepts no input and produces no output.

A $GARBAGE reference is equivalent to a transition that accepts platform specific input and produces as output ε.

Ambiguity

An ambiguous expansion is any expansion for which there is at least one token sequence that when provided as input to the expansion could produce at least two distinct output sequences.

Ambiguous expansions can be constructed of alternatives and repeated expansions plus rule references.

This document does not illustrate all forms of ambiguous expansions but provides examples of some of the form common forms.

A future draft may recommend parsing behavior for ambiguities. The working group notes that the 3rd and 4th principles are vague and thus more difficult to implement consistently.

Examples

Matching a token to a token produces an array of 1 token

Expansion t1
Input t1
Output [t1]

A $NULL reference is matched by an empty input sequence and output is an empty array.

Expansion $NULL
Input ""
Output []

A tag is matched by an empty input sequence and output is an array of 1 tag

Expansion {tag}
Input ""
Output [{tag}]

Concatenation: An expansion consisting of a token and a tag is matched by input containing the token and produces as output a token, tag array.

Expansion t1 {tag1}
Input t1
Output [t1,{tag1}]

Concatenation: an expansion consisting of a sequence of tokens, tags and $NULLs is matching by input that consists of the contained tokens. Output consists of the sequence of tokens and tags with order preserved. e.g.

Expansion t1 $NULL {tag1} t2 {tag2} t3
Input t1 t2 t3
Output [t1,{tag1},t2,{tag2},t3]

Parenthetical structure is not preserved in the result. The following is the same sequence as the previous example but with parentheticals added to the expansion definition.

Expansion ((t1) $NULL) {tag1} (t2 {tag2} t3)
Input t1 t2 t3
Output [t1,{tag1},t2,{tag2},t3]

Alternatives: a set of many alternative tokens is matched by input of a single token and produces as output a single token.

Expansion t1 | t2 |t3
Input t2
Output [t2]

Alternatives: if any single expansion in a set of alternatives can be matched by null input then the set of alternatives may be matched by null input and the output is the output of null-accepting expansion. ($NULL, {tag} and repeat counts of zero all permit null input.)

Expansion t1 | t2 | $NULL
Input ""
Output []

With a different null-accepting expansion:

Expansion t1 | t2 | {tag}
Input ""
Output [{tag}]

Alternatives and ambiguity: several examples of ambiguous expansions with the ambiguity arising from alternatives that accept the same input but produce different output.

Expansion t1 {tag1} | t1 {tag2} | t2
Input t1
Output 1 [t1,{tag1}]
Output 2 [t1,{tag2}]

In this example null input is ambiguous.

Expansion {tag1} | {tag2} | $NULL
Input ""
Output 1 [{tag1}]
Output 2 [{tag2}]
Output 3 []

The following is not ambiguous because the different paths through the expansion produce the same output.

Expansion t1 | t1 | t2
Input t1
Output 1 [t1]
Output 2 [t1]

Repeats: an optional expansion can be either matched by an empty token sequence or by any token sequence that matches the expansion contained within the optional.

Expansion t1 <0-1>
Input 1 ""
Output 1 []
Input 2 t1
Output 2 [t1]

Repeats: order is preserved upon multiple expansions.

Expansion (t1 {tag1}) <0-3>
Input 1 ""
Output 1 []
Input 2 t1
Output 2 [t1,{tag1}]
Input 3 t1,t1,t1
Output 3 [t1,{tag1},t1,{tag1},t1,{tag1}]

Repeats and null input: If the contents of an optional expansion can be matched by an empty input sequence AND the output of matching the contained expansion is always an empty array then the output of matching the optional expansion by an empty sequence is also an empty array.

Expansion $NULL <0-1>
Input ""
Output []

Ambiguous repeats: If a repeated or optional expansion can be matched by an empty input sequence BUT the output of matching the contained expansion may contain tags then the parse is ambiguous. It is recommended that the parse be minimal: Output 1 is preferred.

Expansion {tag} <0->
Input ""
Output 1 []
Output 2 [{tag}]
Output 3 [{tag},{tag}]
Output N [{tag},{tag},{tag},...]

A similar ambiguity arises if the repeated expansion contains a alternative expansion that has a null-accepting expansion.

Expansion (t1 | {tag}) <0-3>
Input t1
Output 1 [t1]
Output 2 [t1,{tag}]
Output 3 [{tag},t1]
Output 4 [t1,{tag},{tag}]
Output 5 [{tag},t1,{tag}]
Output 6 [{tag},{tag},t1]

A sequence with two repeat expansion can be ambiguous if the two repeated expansions can accept the same input but produce different output.

Expansion (t1 {tag1}) <0-2> (t1 {tag2}) <0-2>
Input t1,t1,t1
Output 1 [t1,{tag1},t1,{tag1},t1,{tag1}
Output 2 [t1,{tag1},t1,{tag1},t1,{tag2}
Output 3 [t1,{tag1},t1,{tag2},t1,{tag2}
Output 4 [t1,{tag2},t1,{tag2},t1,{tag2}

H.2 Parsing Rule References

A rule reference is a legal rule expansion (see Section 2.2).

We denote output obtained by matching the token sequence "t1,t2,..." against the expansion $rulename as $rulename[e1,e2,...] where "e1,e2,..." is the entity sequence obtained by matching that token sequence against the rule expansion defined for $rulename.

A distinct structure for matching rule references maintains the parse tree for the result. This structure may be utilized in the semantic interpretation process or other computational processes that derive from the parse output structure.

The next three paragraphs ignore the representation of the URI/alias/root in the result structure. The structure, not the naming, is assumed to be the important characteristic in $rulename[e1,e2,...].

There is no distinction between local rule references (within the same grammar) and external rule references.

There is no distinction between a root reference and a reference to a named grammar.

An alias reference is syntactic convenience for a rule reference and is not distinguished.

Examples

The following is a simple rule reference example.

Rule $x = t1 t2 t3;
Expansion $x
Input t1,t2,t3
Output [$x[t1,t2,t3]]

The following is a rule reference in sequence.

Rule $x = t2 t3 t4;
Expansion t1 $x t5
Input t1,t2,t3,t4,t5
Output [t1,$x[t2,t3,t4],t5]

The following includes a reference to a rule that outputs a tag.

Rule $x = t2 {tag};
Expansion t1 $x t3
Input t1,t2,t3
Output [t1,$x[t2,{tag}],t3]

Multiple references to the same rule are permitted.

Rule $x = t1 {tag1};
Expansion $x $x $x
Input t1,t1,t1
Output [$x[t1,{tag1}],$x[t1,{tag1}],$x[t1,{tag1}]]

Rule references may be repeated.

Rule $x = t1 {tag};
Expansion $x <0->
Input t1,t1,t1
Output [$x[t1,{tag1}],$x[t1,{tag1}],$x[t1,{tag1}]]

H.3 Recursion

The Speech Recognition Grammar Format has the expressive power of a Context Free Grammar. This arises because the language permits a rule to directly or indirectly reference itself. [Note: a Conforming XML Grammar Processor or Conforming ABNF Grammar Processor is not required to support recursive grammars.]

There is no distinct representation for a recursive rule reference.

Examples

Simple right recursion. [This grammar can be written in a non-recursive (regular expression) form.]

Rule $x = t1 {last} | t1 $x;
Expansion $x
Input t1,t1,t1
Output [$x[t1,$x[t1,$x[t1,{last}]]]]

Embedded recursion. Note that this matches any sequence of n t1's followed by n t2's.

Rule $x = {bottom} | (t1 $x t2);
Expansion $x
Input t1,t1,t2,t2
Output [$x[t1,$x[t1,$x[{bottom}],t2],t2]]

Appendix I. Features under Consideration for Future Versions

This appendix is Informative.

The following features are under consideration for versions of the Speech Recognition Grammar Specification after version 1.0: