-
Notifications
You must be signed in to change notification settings - Fork 791
Expand file tree
/
Copy pathOverview.src.html
More file actions
executable file
·3319 lines (2723 loc) · 121 KB
/
Overview.src.html
File metadata and controls
executable file
·3319 lines (2723 loc) · 121 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>CSS Syntax Module Level 3 (CSS3 Syntax)</title>
<link rel=contents href="#contents">
<link rel=index href="#index">
<link rel="stylesheet" type="text/css" href="../default.css">
<link rel="stylesheet" type="text/css"
href="http://www.w3.org/StyleSheets/TR/W3C-[STATUS].css">
<style type="text/css">
/* delete this block when you've cleared out all the .replaceme elements */
.replaceme {
border: solid red;
padding: 0 0.5em;
margin: 0 0.1em;
background: yellow;
color: green;
}
.replaceme:before {
content: "FIXME(";
color: black;
}
.replaceme:after {
content: ")";
color: black;
}
</style>
</head>
<div class="head">
<!--logo-->
<h1>CSS Syntax Module Level 3</h1>
<h2 class="no-num no-toc">[LONGSTATUS] [DATE]</h2>
<dl>
<dt>This version:
<dd>
<a href="http://dev.w3.org/csswg/css3-syntax/">http://dev.w3.org/csswg/css3-syntax/</a>
<!--
<dt>Latest version:
<dd>
<a href="http://www.w3.org/TR/css3-syntax/">http://www.w3.org/TR/css3-syntax/</a>
-->
<dt>Editor's draft:
<dd>
<a href="http://dev.w3.org/csswg/css3-syntax/">http://dev.w3.org/csswg/css3-syntax/</a>
<dt>Previous version:
<dd>
<a href="http://www.w3.org/TR/2003/WD-css3-syntax-20030813/">http://www.w3.org/TR/2003/WD-css3-syntax-20030813/</a>
<dt>Issue Tracking:
<dd>
<a href="https://www.w3.org/Bugs/Public/buglist.cgi?product=CSS&component=Syntax">W3C Bugzilla</a>
<dt>Feedback:
<dd>
<a href="http://lists.w3.org/Archives/Public/www-style/">www-style@w3.org</a>
with subject line “<kbd>[css3-syntax] <var>… message topic …</var></kbd>”
<dt>Editors:
<dd class="h-card vcard">
<a class="p-name fn u-url url" rel="author"
href="http://www.xanthir.com/">Tab Atkins Jr.</a>
(<span class="p-org org">Google, Inc.</span>),
<span class="u-email email">jackalmage@gmail.com</span>
</dl>
<!--copyright-->
<hr title="Separator for header">
</div>
<h2 class="no-num no-toc" id="abstract">Abstract</h2>
<p>
<a href="http://www.w3.org/TR/CSS/">CSS</a> is a language for describing the rendering of structured documents (such as HTML and XML) on screen, on paper, in speech, etc.
This module describes, in general terms, the basic structure and syntax of CSS stylesheets.
It defines, in detail, the syntax and parsing of CSS - how to turn a stream of bytes into a meaningful stylesheet.
<h2 class="no-num no-toc" id="status">Status of this document</h2>
<!--status-->
<p>The following features are at risk: …
<h2 class="no-num no-toc" id="contents">
Table of contents</h2>
<!--toc-->
<h2 id="intro">
Introduction</h2>
<p><em>This section is not normative.</em>
<p><var class="replaceme">Provide background, motivation, etc.</var></p>
<h3 id="placement">
Module interactions</h3>
<p>
This module defines the syntax and parsing of CSS stylesheets.
It supersedes the lexical scanner and grammar defined in CSS 2.1.
<h2 id='syntax-description'>
Description of CSS's Syntax</h2>
<p><em>This section is not normative.</em>
<p>
A CSS document is a series of <i>rules</i>,
which apply CSS properties to elements in the source document,
and <i>at-rules</i>,
which define special processing rules or values for the CSS document.
<p>
A <dfn>rule</dfn> starts with a selector
(defined by the <a href="http://www.w3.org/TR/selectors/">Selectors specification</a>),
then has a {}-wrapped block containing a sequence of declarations.
The selector specifies which elements the declarations will apply to.
Each declaration has a property name,
followed by a colon and the property value,
and finished with a semicolon.
<div class='example'>
<p>
A typical rule might look something like this:
<pre>
p > a {
color: blue;
text-decoration: underline;
}</pre>
<p>
In the above rule, "<code>p > a</code>" is the selector,
which, if the source document is HTML,
selects any <code><a></code> elements that are children of a <code><p></code> element.
<p>
"<code>color: blue;</code>" is a declaration specifying that,
for the elements that match the selector,
their 'color' property should have the value ''blue''.
Similiarly, their 'text-decoration' property should have the value ''underline''.
</div>
<p>
<dfn title="at-rule">At-rules</dfn> are all different, but they have a basic structure in common.
They start with an "@" character followed by their name.
Some <i>at-rules</i> are simple statements,
with their name followed by more CSS values to specify their behavior,
and finally ended by a semicolon.
Others are blocks;
they can have CSS values following their name,
but they end with a {}-wrapped block,
similar to a <i>rule</i>.
Even the contents of these blocks are specific to the given <i>at-rule</i>:
sometimes they contain a sequence of declarations, like a <i>rule</i>;
other times, they may contain additional blocks, or at-rules, or other structures altogether.
<div class='example'>
<p>
Here are several examples of <i>at-rules</i> that illustrate the varied syntax they may contain.
<pre>@import "my-styles.css";</pre>
<p>
The ''@import'' <i>at-rule</i> is a simple statement.
After its name, it takes a single string or ''url()'' function to indicate the stylesheet that it should import.
<pre>
@page :left {
margin-left: 4cm;
margin-right: 3cm;
}</pre>
<p>
The ''@page'' <i>at-rule</i> consists of an optional page selector (the ":left" pseudoclass),
followed by a block of properties that apply to the page when printed.
In this way, it's very similar to a normal <i>rule</i>,
except that its properties don't apply to any "element",
but rather the page itself.
<pre>
@media print {
body { font-size: 10pt }
}</pre>
<p>
The ''@media'' <i>at-rule</i> begins with a media type
and a list of optional media queries.
Its block contains entire rules,
which are only applied when the ''@media''s conditions are fulfilled.
</div>
<p>
Property names and <i>at-rule</i> names are always <b>idents</b>,
which have to start with a letter or a hyphen followed by a letter,
and then can contain letters, numbers, hyphens, or underscores.
You can include any character at all,
even ones that CSS uses in its syntax,
by escaping it with a backslash (\) or by using a hexadecimal escape.
<p>
The syntax of selectors is defined in the <a href="http://www.w3.org/TR/selectors/">Selectors spec</a>.
Similarly, the syntax of the wide variety of CSS values is defined in the <a href="http://www.w3.org/TR/css3-values/">Values & Units spec</a>.
The special syntaxes of individual <i>at-rules</i> can be found in the specs that define them.
<h2>
Tokenizing and Parsing CSS</h2>
<p>
User agents must use the parsing rules described in this section to generate the CSSOM trees from text/css resources.
Together, these rules define what is referred to as the CSS parser.
<p>
This specification defines the parsing rules for CSS documents, whether they are syntactically correct or not.
Certain points in the parsing algorithm are said to be parse errors.
The error handling for parse errors is well-defined:
user agents must either act as described below when encountering such problems,
or must abort processing at the first error that they encounter for which they do not wish to apply the rules described below.
<p>
Conformance checkers must report at least one parse error condition to the user if one or more parse error conditions exist in the document and must not report parse error conditions if none exist in the document.
Conformance checkers may report more than one parse error condition if more than one parse error condition exists in the document.
Conformance checkers are not required to recover from parse errors.
<h3>
Overview of the Parsing Model</h3>
<p>
The input to the CSS parsing process consists of a stream of Unicode code points,
which is passed through a tokenization stage followed by a tree construction stage.
The output is a CSSStyleSheet object.
<p class='note'>
Implementations that do not support scripting do not have to actually create a CSSOM CSSStyleSheet object,
but the CSSOM tree in such cases is still used as the model for the rest of the specification.
<h3>
The input byte stream</h3>
<p>
The stream of Unicode code points that comprises the input to the tokenization stage will be initially seen by the user agent as a stream of bytes
(typically coming over the network or from the local file system).
The bytes encode the actual characters according to a particular character encoding,
which the user agent must use to decode the bytes into characters.
<p>
To decode the stream of bytes into a stream of characters,
UAs must follow these steps.
<p>
The algorithms to <a href="http://encoding.spec.whatwg.org/#concept-encoding-get"><dfn>get an encoding</dfn></a>
and <a href="http://encoding.spec.whatwg.org/#decode"><dfn>decode</dfn></a>
are defined in the <a href="http://encoding.spec.whatwg.org/">Encoding Standard</a>.
<ol>
<li>
Let <var>encoding</var> be utf-8.
<li>
If HTTP or equivalent protocol defines an encoding (e.g. via the charset parameter of the Content-Type header),
<i>get an encoding</i> for the specified value.
If that does not return failure,
set <var>encoding</var> to the return value
and jump to the last step of this algorithm.
<li>
Check the byte stream. If the first several bytes match the hex sequence
<pre>40 63 68 61 72 73 65 74 20 22 (XX)* 22 3B</pre>
then <i>get an encoding</i> for the sequence of XX bytes,
decoded per <code>windows-1252</code>,
and let <var>temp</var> be the return value.
<p class='note'>
Note: Anything ASCII-compatible will do, so using <code>windows-1252</code> is fine.
<p class='note'>
Note: The byte sequence above,
when decoded as ASCII,
is the string "<code>@charset "…";</code>",
where the "…" is the sequence of bytes corresponding to the encoding's name.
<p>
If <var>temp</var> is <code>utf-16</code> or <code>utf-16be</code>,
set <var>temp</var> to <code>utf-8</code>.
If <var>temp</var> is not failure,
set <var>encoding</var> to it
and jump to the last step.
<p class='note'>
This mimics HTML <code><meta></code> behavior.
<li>
<i>Get an encoding</i> for the value of the <code>charset</code> attribute on the <code><link></code> element or <code><?xml-stylesheet?></code> processing instruction that caused the style sheet to be included, if any.
If that does not return failure,
set <var>encoding</var> to the return value
and jump to the last step.
<li>
Set <var>encoding</var> to the encoding of the referring style sheet or document,
if any.
<li>
<i>Decode</i> the byte stream using fallback encoding <var>encoding</var>.
<p class='note'>
Note: the <i>decode</i> algorithm lets the byte order mark (BOM) take precedence,
hence the usage of the term "fallback" above.
</ol>
<p class='issue'>
Anne says that steps 4/5 should be an input to this algorithm from the specs that define importing stylesheet,
to make the algorithm as a whole cleaner.
Perhaps abstract it into the concept of an "environment charset" or something?
<h4>
Preprocessing the input stream</h4>
<p>
The input stream consists of the characters pushed into it as the input byte stream is decoded.
<p>
Before sending the input stream to the tokenizer,
implementations must make the following character substitutions:
<ul>
<li>
Replace any U+000D CARRIAGE RETURN (CR) characters
or pairs of U+000D CARRIAGE RETURN (CR) followed by U+000A LINE FEED (LF)
by a single U+000A LINE FEED (LF) character.
<li>
Replace any U+0000 NULL characters with U+FFFD REPLACEMENT CHARACTER (�).
</ul>
<p>
The <dfn>next input character</dfn> is the first character in the input stream that has not yet been consumed or explicitly ignored by the requirements in this section. Initially, the <i>next input character</i> is the first character in the input. The <dfn>current input character</dfn> is the last character to have been consumed.
<p>
The "EOF" character in the tables below is a conceptual character representing the end of the input stream.
<h3>
Tokenization</h3>
<p>
Implementations must act as if they used the following state machine to tokenize CSS.
The state machine must start in the <i>data state</i>.
Most states consume a single character,
which may have various side-effects,
and either switches the state machine to a new state to reconsume the same character,
or switches it to a new state to consume the next character,
or stays in the same state to consume the next character.
Some states have more complicated behavior and can consume several characters before switching to another state.
<p>
The output of the tokenization step is a series of zero or more of the following tokens:
ident,
function,
at-keyword,
hash,
string,
bad-string,
url,
bad-url,
delim,
number,
percentage,
dimension,
unicode-range,
whitespace,
comment,
cdo,
cdc,
colon,
semicolon,
[,
],
(,
),
{,
}.
<p>
ident, function, at-keyword, hash, string, and url tokens have a value composed of zero or more characters.
Delim tokens have a value composed of a single character.
Number, percentage, and dimension tokens have a representation composed of 1 or more character, a numeric value, and a type flag set to either "integer" or "number". The type flag defaults to "integer" if not otherwise set.
Dimension tokens additionally have a unit composed of one or more characters.
Unicode-range tokens have a range of characters.
<p>
The tokenizer state machine consists of the states defined in the following subsections.
<h4 id='token-diagrams'>
Token Railroad Diagrams</h4>
<p>
<em>This section is non-normative.</em>
<p>
This section presents an informative view of the token parser,
in the form of railroad diagrams.
Railroad diagrams are more compact than a state-machine,
but often easier to read than a regular expression.
<p>
These diagrams are <em>informative</em> and <em>incomplete</em>;
they describe the grammar of "correct" tokens,
but do not describe error-handling at all.
They are provided solely to make it easier to get an intuitive grasp of the syntax of each token.
<dl>
<dt id="ident-diagram">IDENT
<dd>
<!--<script>
Diagram(
Choice(0, Skip(), Terminal('-')),
Choice(0, NonTerminal('a-z A-Z _ or non-ASCII'), NonTerminal('escape')),
ZeroOrMore(
Choice(0, NonTerminal('a-z A-Z 0-9 _ - or non-ASCII'), NonTerminal('escape')))).addTo();
</script>-->
<svg width="831" height="110" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 31 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><g><path d="M40 41h0h.5"></path><path d="M110 41h0h.5"></path><path d="M40 41h20h.5"></path><g><path d="M60 41h30h.5"></path></g><path d="M90 41h20h.5"></path><path d="M40 41a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M60 61h30h.5"></path><rect x="60" y="50" width="30" height="22" rx="10" ry="10"></rect><text x="75" y="65">-</text></g><path d="M90 61a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path></g><g><path d="M110 41h0h.5"></path><path d="M390 41h0h.5"></path><path d="M110 41h20h.5"></path><g><path d="M130 41h240h.5"></path><rect x="130" y="30" width="240" height="22"></rect><text x="250" y="45">a-z A-Z _ or non-ASCII</text></g><path d="M370 41h20h.5"></path><path d="M110 41a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M130 71h240h.5"></path><rect x="210" y="60" width="80" height="22"></rect><text x="250" y="75">escape</text></g><path d="M370 71a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><g><path d="M390 41h0h.5"></path><path d="M790 41h0h.5"></path><path d="M390 41a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M410 21h360h.5"></path></g><path d="M770 21a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M390 41h20h.5"></path><g><path d="M410 41h0h.5"></path><path d="M770 41h0h.5"></path><path d="M410 41h10h.5"></path><g><path d="M420 41h0h.5"></path><path d="M760 41h0h.5"></path><path d="M420 41h20h.5"></path><g><path d="M440 41h300h.5"></path><rect x="440" y="30" width="300" height="22"></rect><text x="590" y="45">a-z A-Z 0-9 _ - or non-ASCII</text></g><path d="M740 41h20h.5"></path><path d="M420 41a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M440 71h300h.5"></path><rect x="550" y="60" width="80" height="22"></rect><text x="590" y="75">escape</text></g><path d="M740 71a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><path d="M760 41h10h.5"></path><path d="M420 41a10 10 0 0 0 -10 10v29a10 10 0 0 0 10 10h.5"></path><g><path d="M420 90h340h.5"></path></g><path d="M760 90a10 10 0 0 0 10 -10v-29a10 10 0 0 0 -10 -10h.5"></path></g><path d="M770 41h20h.5"></path></g><path d="M 790 41 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="function-diagram">FUNCTION
<dd>
<!--<script>
Diagram(NonTerminal('IDENT'), Terminal('(')).addTo();
</script>-->
<svg width="221" height="62" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 21 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><path d="M40 31h10h.5"></path><g><path d="M50 31h70h.5"></path><rect x="50" y="20" width="70" height="22"></rect><text x="85" y="35">IDENT</text></g><path d="M120 31h10h.5"></path><path d="M130 31h10h.5"></path><g><path d="M140 31h30h.5"></path><rect x="140" y="20" width="30" height="22" rx="10" ry="10"></rect><text x="155" y="35">(</text></g><path d="M170 31h10h.5"></path><path d="M 180 31 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="at-keyword-diagram">AT-KEYWORD
<dd>
<!--<script>
Diagram(Terminal('@'), NonTerminal('IDENT')).addTo();
</script>-->
<svg width="221" height="62" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 21 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><path d="M40 31h10h.5"></path><g><path d="M50 31h30h.5"></path><rect x="50" y="20" width="30" height="22" rx="10" ry="10"></rect><text x="65" y="35">@</text></g><path d="M80 31h10h.5"></path><path d="M90 31h10h.5"></path><g><path d="M100 31h70h.5"></path><rect x="100" y="20" width="70" height="22"></rect><text x="135" y="35">IDENT</text></g><path d="M170 31h10h.5"></path><path d="M 180 31 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="hash-diagram">HASH
<dd>
<!--<script>
Diagram(Terminal('#'), OneOrMore(NonTerminal('a-z A-Z 0-9 _ - or non-ASCII'))).addTo();
</script>-->
<svg width="471" height="71" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 21 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><path d="M40 31h10h.5"></path><g><path d="M50 31h30h.5"></path><rect x="50" y="20" width="30" height="22" rx="10" ry="10"></rect><text x="65" y="35">#</text></g><path d="M80 31h10h.5"></path><path d="M90 31h10h.5"></path><g><path d="M100 31h0h.5"></path><path d="M420 31h0h.5"></path><path d="M100 31h10h.5"></path><g><path d="M110 31h300h.5"></path><rect x="110" y="20" width="300" height="22"></rect><text x="260" y="35">a-z A-Z 0-9 _ - or non-ASCII</text></g><path d="M410 31h10h.5"></path><path d="M110 31a10 10 0 0 0 -10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M110 51h300h.5"></path></g><path d="M410 51a10 10 0 0 0 10 -10v0a10 10 0 0 0 -10 -10h.5"></path></g><path d="M420 31h10h.5"></path><path d="M 430 31 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="string-diagram">STRING
<dd>
<!--<script>
Diagram(
Choice(0,
Sequence(
Terminal('"'),
ZeroOrMore(
Choice(0,
NonTerminal('not " or \\'),
NonTerminal('escape'))),
Terminal('"')),
Sequence(
Terminal('\''),
ZeroOrMore(
Choice(0,
NonTerminal("not \' or \\"),
NonTerminal('escape'))),
Terminal('\'')))).addTo();
</script>-->
<svg width="441" height="188" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 31 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><g><path d="M40 41h0h.5"></path><path d="M400 41h0h.5"></path><path d="M40 41h20h.5"></path><g><path d="M60 41h0h.5"></path><path d="M60 41h10h.5"></path><g><path d="M70 41h30h.5"></path><rect x="70" y="30" width="30" height="22" rx="10" ry="10"></rect><text x="85" y="45">"</text></g><path d="M100 41h10h.5"></path><g><path d="M110 41h0h.5"></path><path d="M330 41h0h.5"></path><path d="M110 41a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M130 21h180h.5"></path></g><path d="M310 21a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M110 41h20h.5"></path><g><path d="M130 41h0h.5"></path><path d="M310 41h0h.5"></path><path d="M130 41h10h.5"></path><g><path d="M140 41h0h.5"></path><path d="M300 41h0h.5"></path><path d="M140 41h20h.5"></path><g><path d="M160 41h120h.5"></path><rect x="160" y="30" width="120" height="22"></rect><text x="220" y="45">not " or \</text></g><path d="M280 41h20h.5"></path><path d="M140 41a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M160 71h120h.5"></path><rect x="180" y="60" width="80" height="22"></rect><text x="220" y="75">escape</text></g><path d="M280 71a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><path d="M300 41h10h.5"></path><path d="M140 41a10 10 0 0 0 -10 10v29a10 10 0 0 0 10 10h.5"></path><g><path d="M140 90h160h.5"></path></g><path d="M300 90a10 10 0 0 0 10 -10v-29a10 10 0 0 0 -10 -10h.5"></path></g><path d="M310 41h20h.5"></path></g><path d="M330 41h10h.5"></path><g><path d="M340 41h30h.5"></path><rect x="340" y="30" width="30" height="22" rx="10" ry="10"></rect><text x="355" y="45">"</text></g><path d="M370 41h10h.5"></path><path d="M380 41h0h.5"></path></g><path d="M380 41h20h.5"></path><path d="M40 41a10 10 0 0 1 10 10v58a10 10 0 0 0 10 10h.5"></path><g><path d="M60 119h0h.5"></path><path d="M60 119h10h.5"></path><g><path d="M70 119h30h.5"></path><rect x="70" y="108" width="30" height="22" rx="10" ry="10"></rect><text x="85" y="123">'</text></g><path d="M100 119h10h.5"></path><g><path d="M110 119h0h.5"></path><path d="M330 119h0h.5"></path><path d="M110 119a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M130 99h180h.5"></path></g><path d="M310 99a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M110 119h20h.5"></path><g><path d="M130 119h0h.5"></path><path d="M310 119h0h.5"></path><path d="M130 119h10h.5"></path><g><path d="M140 119h0h.5"></path><path d="M300 119h0h.5"></path><path d="M140 119h20h.5"></path><g><path d="M160 119h120h.5"></path><rect x="160" y="108" width="120" height="22"></rect><text x="220" y="123">not ' or \</text></g><path d="M280 119h20h.5"></path><path d="M140 119a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M160 149h120h.5"></path><rect x="180" y="138" width="80" height="22"></rect><text x="220" y="153">escape</text></g><path d="M280 149a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><path d="M300 119h10h.5"></path><path d="M140 119a10 10 0 0 0 -10 10v29a10 10 0 0 0 10 10h.5"></path><g><path d="M140 168h160h.5"></path></g><path d="M300 168a10 10 0 0 0 10 -10v-29a10 10 0 0 0 -10 -10h.5"></path></g><path d="M310 119h20h.5"></path></g><path d="M330 119h10h.5"></path><g><path d="M340 119h30h.5"></path><rect x="340" y="108" width="30" height="22" rx="10" ry="10"></rect><text x="355" y="123">'</text></g><path d="M370 119h10h.5"></path><path d="M380 119h0h.5"></path></g><path d="M380 119a10 10 0 0 0 10 -10v-58a10 10 0 0 1 10 -10h.5"></path></g><path d="M 400 41 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="url-diagram">URL
<dd>
<!--<script>
Diagram(
Choice(0, Terminal('u'), Terminal('U')),
Choice(0, Terminal('r'), Terminal('R')),
Choice(0, Terminal('l'), Terminal('L')),
Terminal('('),
Choice(1,
Optional(NonTerminal('WS')),
Sequence(
Optional(NonTerminal('WS')),
NonTerminal('STRING'),
Optional(NonTerminal('WS'))),
Sequence(
Optional(NonTerminal('WS')),
OneOrMore(
Choice(0,
NonTerminal('not " \' ( ) WS or NPC'),
NonTerminal('escape'))),
Optional(NonTerminal('WS')))),
Terminal(')')).addTo();
</script>-->
<svg width="901" height="190" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 71 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><g><path d="M40 81h0h.5"></path><path d="M110 81h0h.5"></path><path d="M40 81h20h.5"></path><g><path d="M60 81h30h.5"></path><rect x="60" y="70" width="30" height="22" rx="10" ry="10"></rect><text x="75" y="85">u</text></g><path d="M90 81h20h.5"></path><path d="M40 81a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M60 111h30h.5"></path><rect x="60" y="100" width="30" height="22" rx="10" ry="10"></rect><text x="75" y="115">U</text></g><path d="M90 111a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><g><path d="M110 81h0h.5"></path><path d="M180 81h0h.5"></path><path d="M110 81h20h.5"></path><g><path d="M130 81h30h.5"></path><rect x="130" y="70" width="30" height="22" rx="10" ry="10"></rect><text x="145" y="85">r</text></g><path d="M160 81h20h.5"></path><path d="M110 81a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M130 111h30h.5"></path><rect x="130" y="100" width="30" height="22" rx="10" ry="10"></rect><text x="145" y="115">R</text></g><path d="M160 111a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><g><path d="M180 81h0h.5"></path><path d="M250 81h0h.5"></path><path d="M180 81h20h.5"></path><g><path d="M200 81h30h.5"></path><rect x="200" y="70" width="30" height="22" rx="10" ry="10"></rect><text x="215" y="85">l</text></g><path d="M230 81h20h.5"></path><path d="M180 81a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M200 111h30h.5"></path><rect x="200" y="100" width="30" height="22" rx="10" ry="10"></rect><text x="215" y="115">L</text></g><path d="M230 111a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><path d="M250 81h10h.5"></path><g><path d="M260 81h30h.5"></path><rect x="260" y="70" width="30" height="22" rx="10" ry="10"></rect><text x="275" y="85">(</text></g><path d="M290 81h10h.5"></path><g><path d="M300 81h0h.5"></path><path d="M810 81h0h.5"></path><path d="M300 81a10 10 0 0 0 10 -10v-20a10 10 0 0 1 10 -10h.5"></path><g><path d="M320 41h195h.5"></path><path d="M595 41h195h.5"></path><path d="M515 41a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M535 21h40h.5"></path></g><path d="M575 21a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M515 41h20h.5"></path><g><path d="M535 41h40h.5"></path><rect x="535" y="30" width="40" height="22"></rect><text x="555" y="45">WS</text></g><path d="M575 41h20h.5"></path></g><path d="M790 41a10 10 0 0 1 10 10v20a10 10 0 0 0 10 10h.5"></path><path d="M300 81h20h.5"></path><g><path d="M320 81h105h.5"></path><g><path d="M425 81h0h.5"></path><path d="M505 81h0h.5"></path><path d="M425 81a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M445 61h40h.5"></path></g><path d="M485 61a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M425 81h20h.5"></path><g><path d="M445 81h40h.5"></path><rect x="445" y="70" width="40" height="22"></rect><text x="465" y="85">WS</text></g><path d="M485 81h20h.5"></path></g><path d="M505 81h10h.5"></path><g><path d="M515 81h80h.5"></path><rect x="515" y="70" width="80" height="22"></rect><text x="555" y="85">STRING</text></g><path d="M595 81h10h.5"></path><g><path d="M605 81h0h.5"></path><path d="M685 81h0h.5"></path><path d="M605 81a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M625 61h40h.5"></path></g><path d="M665 61a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M605 81h20h.5"></path><g><path d="M625 81h40h.5"></path><rect x="625" y="70" width="40" height="22"></rect><text x="645" y="85">WS</text></g><path d="M665 81h20h.5"></path></g><path d="M685 81h105h.5"></path></g><path d="M790 81h20h.5"></path><path d="M300 81a10 10 0 0 1 10 10v20a10 10 0 0 0 10 10h.5"></path><g><path d="M320 121h0h.5"></path><g><path d="M320 121h0h.5"></path><path d="M400 121h0h.5"></path><path d="M320 121a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M340 101h40h.5"></path></g><path d="M380 101a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M320 121h20h.5"></path><g><path d="M340 121h40h.5"></path><rect x="340" y="110" width="40" height="22"></rect><text x="360" y="125">WS</text></g><path d="M380 121h20h.5"></path></g><path d="M400 121h10h.5"></path><g><path d="M410 121h0h.5"></path><path d="M700 121h0h.5"></path><path d="M410 121h10h.5"></path><g><path d="M420 121h0h.5"></path><path d="M690 121h0h.5"></path><path d="M420 121h20h.5"></path><g><path d="M440 121h230h.5"></path><rect x="440" y="110" width="230" height="22"></rect><text x="555" y="125">not " ' ( ) WS or NPC</text></g><path d="M670 121h20h.5"></path><path d="M420 121a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M440 151h230h.5"></path><rect x="515" y="140" width="80" height="22"></rect><text x="555" y="155">escape</text></g><path d="M670 151a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><path d="M690 121h10h.5"></path><path d="M420 121a10 10 0 0 0 -10 10v29a10 10 0 0 0 10 10h.5"></path><g><path d="M420 170h270h.5"></path></g><path d="M690 170a10 10 0 0 0 10 -10v-29a10 10 0 0 0 -10 -10h.5"></path></g><path d="M700 121h10h.5"></path><g><path d="M710 121h0h.5"></path><path d="M790 121h0h.5"></path><path d="M710 121a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M730 101h40h.5"></path></g><path d="M770 101a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M710 121h20h.5"></path><g><path d="M730 121h40h.5"></path><rect x="730" y="110" width="40" height="22"></rect><text x="750" y="125">WS</text></g><path d="M770 121h20h.5"></path></g><path d="M790 121h0h.5"></path></g><path d="M790 121a10 10 0 0 0 10 -10v-20a10 10 0 0 1 10 -10h.5"></path></g><path d="M810 81h10h.5"></path><g><path d="M820 81h30h.5"></path><rect x="820" y="70" width="30" height="22" rx="10" ry="10"></rect><text x="835" y="85">)</text></g><path d="M850 81h10h.5"></path><path d="M 860 81 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="number-diagram">NUMBER
<dd>
<!--<script>
Diagram(
Choice(1, Terminal('+'), Skip(), Terminal('-')),
Choice(0,
Sequence(
OneOrMore(NonTerminal('digit')),
Terminal('.'),
OneOrMore(NonTerminal('digit'))),
OneOrMore(NonTerminal('digit')),
Sequence(
Terminal('.'),
OneOrMore(NonTerminal('digit')))),
Choice(0,
Skip(),
Sequence(
Choice(0, Terminal('e'), Terminal('E')),
Choice(1, Terminal('+'), Skip(), Terminal('-')),
OneOrMore(NonTerminal('digit'))))).addTo();
</script>-->
<svg width="751" height="179" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 50 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><g><path d="M40 60h0h.5"></path><path d="M110 60h0h.5"></path><path d="M40 60a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M60 40h30h.5"></path><rect x="60" y="29" width="30" height="22" rx="10" ry="10"></rect><text x="75" y="44">+</text></g><path d="M90 40a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M40 60h20h.5"></path><g><path d="M60 60h30h.5"></path></g><path d="M90 60h20h.5"></path><path d="M40 60a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M60 80h30h.5"></path><rect x="60" y="69" width="30" height="22" rx="10" ry="10"></rect><text x="75" y="84">-</text></g><path d="M90 80a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path></g><g><path d="M110 60h0h.5"></path><path d="M420 60h0h.5"></path><path d="M110 60h20h.5"></path><g><path d="M130 60h0h.5"></path><path d="M130 60h10h.5"></path><g><path d="M140 60h0h.5"></path><path d="M230 60h0h.5"></path><path d="M140 60h10h.5"></path><g><path d="M150 60h70h.5"></path><rect x="150" y="49" width="70" height="22"></rect><text x="185" y="64">digit</text></g><path d="M220 60h10h.5"></path><path d="M150 60a10 10 0 0 0 -10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M150 80h70h.5"></path></g><path d="M220 80a10 10 0 0 0 10 -10v0a10 10 0 0 0 -10 -10h.5"></path></g><path d="M230 60h10h.5"></path><path d="M240 60h10h.5"></path><g><path d="M250 60h30h.5"></path><rect x="250" y="49" width="30" height="22" rx="10" ry="10"></rect><text x="265" y="64">.</text></g><path d="M280 60h10h.5"></path><path d="M290 60h10h.5"></path><g><path d="M300 60h0h.5"></path><path d="M390 60h0h.5"></path><path d="M300 60h10h.5"></path><g><path d="M310 60h70h.5"></path><rect x="310" y="49" width="70" height="22"></rect><text x="345" y="64">digit</text></g><path d="M380 60h10h.5"></path><path d="M310 60a10 10 0 0 0 -10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M310 80h70h.5"></path></g><path d="M380 80a10 10 0 0 0 10 -10v0a10 10 0 0 0 -10 -10h.5"></path></g><path d="M390 60h10h.5"></path><path d="M400 60h0h.5"></path></g><path d="M400 60h20h.5"></path><path d="M110 60a10 10 0 0 1 10 10v19a10 10 0 0 0 10 10h.5"></path><g><path d="M130 99h90h.5"></path><path d="M310 99h90h.5"></path><path d="M220 99h10h.5"></path><g><path d="M230 99h70h.5"></path><rect x="230" y="88" width="70" height="22"></rect><text x="265" y="103">digit</text></g><path d="M300 99h10h.5"></path><path d="M230 99a10 10 0 0 0 -10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M230 119h70h.5"></path></g><path d="M300 119a10 10 0 0 0 10 -10v0a10 10 0 0 0 -10 -10h.5"></path></g><path d="M400 99a10 10 0 0 0 10 -10v-19a10 10 0 0 1 10 -10h.5"></path><path d="M110 60a10 10 0 0 1 10 10v58a10 10 0 0 0 10 10h.5"></path><g><path d="M130 138h55h.5"></path><path d="M185 138h10h.5"></path><g><path d="M195 138h30h.5"></path><rect x="195" y="127" width="30" height="22" rx="10" ry="10"></rect><text x="210" y="142">.</text></g><path d="M225 138h10h.5"></path><path d="M235 138h10h.5"></path><g><path d="M245 138h0h.5"></path><path d="M335 138h0h.5"></path><path d="M245 138h10h.5"></path><g><path d="M255 138h70h.5"></path><rect x="255" y="127" width="70" height="22"></rect><text x="290" y="142">digit</text></g><path d="M325 138h10h.5"></path><path d="M255 138a10 10 0 0 0 -10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M255 158h70h.5"></path></g><path d="M325 158a10 10 0 0 0 10 -10v0a10 10 0 0 0 -10 -10h.5"></path></g><path d="M335 138h10h.5"></path><path d="M345 138h55h.5"></path></g><path d="M400 138a10 10 0 0 0 10 -10v-58a10 10 0 0 1 10 -10h.5"></path></g><g><path d="M420 60h0h.5"></path><path d="M710 60h0h.5"></path><path d="M420 60h20h.5"></path><g><path d="M440 60h250h.5"></path></g><path d="M690 60h20h.5"></path><path d="M420 60a10 10 0 0 1 10 10v28a10 10 0 0 0 10 10h.5"></path><g><path d="M440 108h0h.5"></path><g><path d="M440 108h0h.5"></path><path d="M510 108h0h.5"></path><path d="M440 108h20h.5"></path><g><path d="M460 108h30h.5"></path><rect x="460" y="97" width="30" height="22" rx="10" ry="10"></rect><text x="475" y="112">e</text></g><path d="M490 108h20h.5"></path><path d="M440 108a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M460 138h30h.5"></path><rect x="460" y="127" width="30" height="22" rx="10" ry="10"></rect><text x="475" y="142">E</text></g><path d="M490 138a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><g><path d="M510 108h0h.5"></path><path d="M580 108h0h.5"></path><path d="M510 108a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M530 88h30h.5"></path><rect x="530" y="77" width="30" height="22" rx="10" ry="10"></rect><text x="545" y="92">+</text></g><path d="M560 88a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M510 108h20h.5"></path><g><path d="M530 108h30h.5"></path></g><path d="M560 108h20h.5"></path><path d="M510 108a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M530 128h30h.5"></path><rect x="530" y="117" width="30" height="22" rx="10" ry="10"></rect><text x="545" y="132">-</text></g><path d="M560 128a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path></g><path d="M580 108h10h.5"></path><g><path d="M590 108h0h.5"></path><path d="M680 108h0h.5"></path><path d="M590 108h10h.5"></path><g><path d="M600 108h70h.5"></path><rect x="600" y="97" width="70" height="22"></rect><text x="635" y="112">digit</text></g><path d="M670 108h10h.5"></path><path d="M600 108a10 10 0 0 0 -10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M600 128h70h.5"></path></g><path d="M670 128a10 10 0 0 0 10 -10v0a10 10 0 0 0 -10 -10h.5"></path></g><path d="M680 108h10h.5"></path><path d="M690 108h0h.5"></path></g><path d="M690 108a10 10 0 0 0 10 -10v-28a10 10 0 0 1 10 -10h.5"></path></g><path d="M 710 60 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="dimension-diagram">DIMENSION
<dd>
<!--<script>
Diagram(NonTerminal('NUMBER'), NonTerminal('IDENT')).addTo();
</script>-->
<svg width="271" height="62" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 21 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><path d="M40 31h10h.5"></path><g><path d="M50 31h80h.5"></path><rect x="50" y="20" width="80" height="22"></rect><text x="90" y="35">NUMBER</text></g><path d="M130 31h10h.5"></path><path d="M140 31h10h.5"></path><g><path d="M150 31h70h.5"></path><rect x="150" y="20" width="70" height="22"></rect><text x="185" y="35">IDENT</text></g><path d="M220 31h10h.5"></path><path d="M 230 31 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="percentage-diagram">PERCENTAGE
<dd>
<!--<script>
Diagram(NonTerminal('NUMBER'), Terminal('%')).addTo();
</script>-->
<svg width="231" height="62" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 21 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><path d="M40 31h10h.5"></path><g><path d="M50 31h80h.5"></path><rect x="50" y="20" width="80" height="22"></rect><text x="90" y="35">NUMBER</text></g><path d="M130 31h10h.5"></path><path d="M140 31h10h.5"></path><g><path d="M150 31h30h.5"></path><rect x="150" y="20" width="30" height="22" rx="10" ry="10"></rect><text x="165" y="35">%</text></g><path d="M180 31h10h.5"></path><path d="M 190 31 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="unicode-range-diagram">UNICODE-RANGE
<dd>
<!--<script>
Diagram(
Choice(0,
Terminal('U'),
Terminal('u')),
Terminal('+'),
Choice(0,
Sequence(OneOrMore(NonTerminal('hex digit'), Comment('1-6 times'))),
Sequence(
ZeroOrMore(NonTerminal('hex digit'), Comment('1-5 times')),
OneOrMore(Terminal('?'), Comment('1 to (6 - digits) times'))),
Sequence(
OneOrMore(NonTerminal('hex digit'), Comment('1-6 times')),
Terminal('-'),
OneOrMore(NonTerminal('hex digit'), Comment('1-6 times'))))).addTo();
</script>-->
<svg width="622" height="222" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 21 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><g><path d="M40 31h0h.5"></path><path d="M110 31h0h.5"></path><path d="M40 31h20h.5"></path><g><path d="M60 31h30h.5"></path><rect x="60" y="20" width="30" height="22" rx="10" ry="10"></rect><text x="75" y="35">U</text></g><path d="M90 31h20h.5"></path><path d="M40 31a10 10 0 0 1 10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M60 61h30h.5"></path><rect x="60" y="50" width="30" height="22" rx="10" ry="10"></rect><text x="75" y="65">u</text></g><path d="M90 61a10 10 0 0 0 10 -10v-10a10 10 0 0 1 10 -10h.5"></path></g><path d="M110 31h10h.5"></path><g><path d="M120 31h30h.5"></path><rect x="120" y="20" width="30" height="22" rx="10" ry="10"></rect><text x="135" y="35">+</text></g><path d="M150 31h10h.5"></path><g><path d="M160 31h0h.5"></path><path d="M581 31h0h.5"></path><path d="M160 31h20h.5"></path><g><path d="M180 31h115.5h.5"></path><path d="M295.5 31h10h.5"></path><g><path d="M305.5 31h0h.5"></path><path d="M435.5 31h0h.5"></path><path d="M305.5 31h10h.5"></path><g><path d="M315.5 31h110h.5"></path><rect x="315.5" y="20" width="110" height="22"></rect><text x="370.5" y="35">hex digit</text></g><path d="M425.5 31h10h.5"></path><path d="M315.5 31a10 10 0 0 0 -10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M315.5 61h18.5h.5"></path><path d="M407 61h18.5h.5"></path><text x="370.5" y="66" class="comment">1-6 times</text></g><path d="M425.5 61a10 10 0 0 0 10 -10v-10a10 10 0 0 0 -10 -10h.5"></path></g><path d="M435.5 31h10h.5"></path><path d="M445.5 31h115.5h.5"></path></g><path d="M561 31h20h.5"></path><path d="M160 31a10 10 0 0 1 10 10v50a10 10 0 0 0 10 10h.5"></path><g><path d="M180 101h0h.5"></path><g><path d="M180 101h0h.5"></path><path d="M350 101h0h.5"></path><path d="M180 101a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M200 81h130h.5"></path></g><path d="M330 81a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M180 101h20h.5"></path><g><path d="M200 101h0h.5"></path><path d="M330 101h0h.5"></path><path d="M200 101h10h.5"></path><g><path d="M210 101h110h.5"></path><rect x="210" y="90" width="110" height="22"></rect><text x="265" y="105">hex digit</text></g><path d="M320 101h10h.5"></path><path d="M210 101a10 10 0 0 0 -10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M210 131h18.5h.5"></path><path d="M301.5 131h18.5h.5"></path><text x="265" y="136" class="comment">1-5 times</text></g><path d="M320 131a10 10 0 0 0 10 -10v-10a10 10 0 0 0 -10 -10h.5"></path></g><path d="M330 101h20h.5"></path></g><path d="M350 101h10h.5"></path><g><path d="M360 101h0h.5"></path><path d="M551 101h0h.5"></path><path d="M360 101h10h.5"></path><g><path d="M370 101h171h.5"></path><rect x="440.5" y="90" width="30" height="22" rx="10" ry="10"></rect><text x="455.5" y="105">?</text></g><path d="M541 101h10h.5"></path><path d="M370 101a10 10 0 0 0 -10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M370 131h0h.5"></path><path d="M541 131h0h.5"></path><text x="455.5" y="136" class="comment">1 to (6 - digits) times</text></g><path d="M541 131a10 10 0 0 0 10 -10v-10a10 10 0 0 0 -10 -10h.5"></path></g><path d="M551 101h10h.5"></path><path d="M561 101h0h.5"></path></g><path d="M561 101a10 10 0 0 0 10 -10v-50a10 10 0 0 1 10 -10h.5"></path><path d="M160 31a10 10 0 0 1 10 10v110a10 10 0 0 0 10 10h.5"></path><g><path d="M180 161h15.5h.5"></path><path d="M195.5 161h10h.5"></path><g><path d="M205.5 161h0h.5"></path><path d="M335.5 161h0h.5"></path><path d="M205.5 161h10h.5"></path><g><path d="M215.5 161h110h.5"></path><rect x="215.5" y="150" width="110" height="22"></rect><text x="270.5" y="165">hex digit</text></g><path d="M325.5 161h10h.5"></path><path d="M215.5 161a10 10 0 0 0 -10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M215.5 191h18.5h.5"></path><path d="M307 191h18.5h.5"></path><text x="270.5" y="196" class="comment">1-6 times</text></g><path d="M325.5 191a10 10 0 0 0 10 -10v-10a10 10 0 0 0 -10 -10h.5"></path></g><path d="M335.5 161h10h.5"></path><path d="M345.5 161h10h.5"></path><g><path d="M355.5 161h30h.5"></path><rect x="355.5" y="150" width="30" height="22" rx="10" ry="10"></rect><text x="370.5" y="165">-</text></g><path d="M385.5 161h10h.5"></path><path d="M395.5 161h10h.5"></path><g><path d="M405.5 161h0h.5"></path><path d="M535.5 161h0h.5"></path><path d="M405.5 161h10h.5"></path><g><path d="M415.5 161h110h.5"></path><rect x="415.5" y="150" width="110" height="22"></rect><text x="470.5" y="165">hex digit</text></g><path d="M525.5 161h10h.5"></path><path d="M415.5 161a10 10 0 0 0 -10 10v10a10 10 0 0 0 10 10h.5"></path><g><path d="M415.5 191h18.5h.5"></path><path d="M507 191h18.5h.5"></path><text x="470.5" y="196" class="comment">1-6 times</text></g><path d="M525.5 191a10 10 0 0 0 10 -10v-10a10 10 0 0 0 -10 -10h.5"></path></g><path d="M535.5 161h10h.5"></path><path d="M545.5 161h15.5h.5"></path></g><path d="M561 161a10 10 0 0 0 10 -10v-110a10 10 0 0 1 10 -10h.5"></path></g><path d="M 581 31 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="comment-diagram">COMMENT
<dd>
<!--<script>
Diagram(
Terminal('/*'),
ZeroOrMore(
NonTerminal("anything but * followed by /")),
Terminal('*/')).addTo();
</script>-->
<svg width="561" height="81" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 31 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><path d="M40 41h10h.5"></path><g><path d="M50 41h40h.5"></path><rect x="50" y="30" width="40" height="22" rx="10" ry="10"></rect><text x="70" y="45">/*</text></g><path d="M90 41h10h.5"></path><g><path d="M100 41h0h.5"></path><path d="M460 41h0h.5"></path><path d="M100 41a10 10 0 0 0 10 -10v0a10 10 0 0 1 10 -10h.5"></path><g><path d="M120 21h320h.5"></path></g><path d="M440 21a10 10 0 0 1 10 10v0a10 10 0 0 0 10 10h.5"></path><path d="M100 41h20h.5"></path><g><path d="M120 41h0h.5"></path><path d="M440 41h0h.5"></path><path d="M120 41h10h.5"></path><g><path d="M130 41h300h.5"></path><rect x="130" y="30" width="300" height="22"></rect><text x="280" y="45">anything but * followed by /</text></g><path d="M430 41h10h.5"></path><path d="M130 41a10 10 0 0 0 -10 10v0a10 10 0 0 0 10 10h.5"></path><g><path d="M130 61h300h.5"></path></g><path d="M430 61a10 10 0 0 0 10 -10v0a10 10 0 0 0 -10 -10h.5"></path></g><path d="M440 41h20h.5"></path></g><path d="M460 41h10h.5"></path><g><path d="M470 41h40h.5"></path><rect x="470" y="30" width="40" height="22" rx="10" ry="10"></rect><text x="490" y="45">*/</text></g><path d="M510 41h10h.5"></path><path d="M 520 41 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="cdo-diagram">CDO
<dd>
<!--<script>
Diagram(Terminal("<" + "!--")).addTo();
</script>-->
<svg width="161" height="62" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 21 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><path d="M40 31h10h.5"></path><g><path d="M50 31h60h.5"></path><rect x="50" y="20" width="60" height="22" rx="10" ry="10"></rect><text x="80" y="35"><!--</text></g><path d="M110 31h10h.5"></path><path d="M 120 31 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
<dt id="cdc-diagram">CDC
<dd>
<!--<script>
Diagram(Terminal("-" + "->")).addTo();
</script>-->
<svg width="151" height="62" class="diagram"><style>.diagram{background-color:hsl(30,20%,95%);} .diagram path{stroke-width:3;stroke:black;fill:rgba(0,0,0,0);} .diagram text{font:bold 14px monospace;text-anchor:middle;} .diagram text.label{text-anchor:start;} .diagram text.comment{font:italic 12px monospace;} .diagram rect{stroke-width:3;stroke:black;fill:hsl(120,100%,90%);}</style><g transform="translate(.5 .5)"><path d="M 20 21 v 20 m 10 -20 v 20 m -10 -10 h 20.5"></path><path d="M40 31h10h.5"></path><g><path d="M50 31h50h.5"></path><rect x="50" y="20" width="50" height="22" rx="10" ry="10"></rect><text x="75" y="35">--></text></g><path d="M100 31h10h.5"></path><path d="M 110 31 h 20 m -10 -10 v 20 m 10 -20 v 20"></path></g></svg>
</dl>
<h4>
Tokenizer Flags</h4>
<p>
The tokenizer can be run with any of several flags that alter its behavior.
<dl>
<dt>the <dfn>transform function whitespace</dfn> flag
<dd>
This flag is set when parsing SVG's <code>transform</code> attribute.
When this is set, whitespace is allowed between the name of a transform function and its opening parenthesis.
<dt>the <dfn>quirks mode</dfn> flag
<dd>
This flag is set when the host document is in quirks mode.
When this is set,
the parser allows <a href="#consume-a-primitive-with-the-hashless-co">hash colors without the # character</a>
and <a href="#consume-a-primitive-with-the-unitless-le">pixel lengths without the 'px' unit</a>
in some circumstances.
</dl>
<h4>
Definitions</h4>
<p>
This section defines several terms used during the tokenization phase.
<dl>
<dt><dfn>digit</dfn>
<dd>
A character between U+0030 DIGIT ZERO (0) and U+0039 DIGIT NINE (9).
<dt><dfn>hex digit</dfn>
<dd>
A <i>digit</i>,
or a character between U+0041 LATIN CAPITAL LETTER A (A) and U+0046 LATIN CAPITAL LETTER F (F),
or a character between U+0061 LATIN SMALL LETTER A (a) and U+0066 LATIN SMALL LETTER F (f).
<dt><dfn>uppercase letter</dfn>
<dd>
A character between U+0041 LATIN CAPITAL LETTER A (A) and U+005A LATIN CAPITAL LETTER Z (Z).
<dt><dfn>lowercase letter</dfn>
<dd>
A character between U+0061 LATIN SMALL LETTER A (a) and U+007A LATIN SMALL LETTER Z (z).
<dt><dfn>letter</dfn>
<dd>
An <i>uppercase letter</i>
or a <i>lowercase letter</i>.
<dt><dfn>non-ASCII character</dfn>
<dd>
A character with a codepoint equal to or greater than U+00A0 NO-BREAK SPACE.
<dt><dfn>name-start character</dfn>
<dd>
A <i>letter</i>,
a <i>non-ASCII character</i>,
or U+005F LOW LINE (_).
<dt><dfn>name character</dfn>
<dd>
A <i>name-start character</i>,
A <i>digit</i>,
or U+002D HYPHEN-MINUS (-).
<dt><dfn>non-printable character</dfn>
<dd>
A character between U+0000 NULL and U+0008 BACKSPACE
or a character between U+000E SHIFT OUT and U+001F INFORMATION SEPARATOR ONE
or a character between U+007F DELETE and U+009F APPLICATION PROGRAM COMMAND.
<dt><dfn>newline</dfn>
<dd>
U+000A LINE FEED or U+000C FORM FEED.
<span class='note'>
Note that U+000D CARRIAGE RETURN is not included in this definition,
as it is removed from the stream during preprocessing.
</span>
<dt><dfn>whitespace</dfn>
<dd>A <i>newline</i>, U+0009 CHARACTER TABULATION, or U+0020 SPACE.
<dt><dfn>maximum allowed codepoint</dfn>
<dd>The greatest codepoint defined by Unicode. This is currently U+10FFFF.
</dl>
<h4>
<dfn>Data state</dfn></h4>
<p>
Consume the <i>next input character</i>.
<dl>
<dt><i>whitespace</i>
<dd>
Consume as much <i>whitespace</i> as possible.
Emit a whitespace token.
Remain in this state.
<dt>U+0022 QUOTATION MARK (")
<dd>
Switch to the <i>double-quote-string state</i>.
<dt>U+0023 NUMBER SIGN (#)
<dd>
Switch to the <i>hash state</i>.
<dt>U+0027 APOSTROPHE (')
<dd>
Switch to the <i>single-quote-string state</i>.
<dt>U+0028 LEFT PARENTHESIS (()
<dd>
Emit a ( token.
Remain in this state.
<dt>U+0029 RIGHT PARENTHESIS ())
<dd>
Emit a ) token.
Remain in this state.
<dt>U+002B PLUS SIGN (+)
<dd>
If the <i>next input character</i> is
a <i>digit</i>,
or the <i title="next input character">next 2 input characters</i> are
U+002E FULL STOP (.)
followed by a <i>digit</i>,
switch to the <i>number state</i>.
Reconsume the <i>current input character</i>.
<p>
Otherwise,
emit a delim token
with its value set to U+002B PLUS SIGN (+).
Remain in this state.
<dt>U+002D HYPHEN-MINUS (-)
<dd>
If the <i title="next input character">next 2 input characters</i> are
U+002D HYPHEN-MINUS
U+003E GREATER-THAN SIGN
(->),
consume them,
emit a CDC token,
and remain in this state.
<p>
Otherwise,
if the <i>next input character</i> is
a <i>digit</i>,
or the <i title="next input character">next 2 input characters</i> are
U+002E FULL STOP (.)
followed by a <i>digit</i>,
switch to the <i>number state</i>.
Reconsume the <i>current input character</i>.
<p>
Otherwise,
if the <i>next input character</i> is
a <i>name-start character</i>,
switch to the <i>ident state</i>.
Reconsume the <i>current input character</i>.
<p>
Otherwise,
emit a delim token
with its value set to U+002D HYPHEN-MINUS (-).
Remain in this state.
<dt>U+002E FULL STOP (.)
<dd>
If the <i>next input character</i> is a digit,
switch to the <i>number state</i>.
Reconsume the <i>current input character</i>.
<p>
Otherwise,
emit a delim token
with its value set to U+002E FULL STOP (.).
Remain in this state.
<dt>U+002F SOLIDUS (/)
<dd>
If the <i>next input character</i> is U+002A ASTERISK (*),
consume it
and switch to the <i>comment state</i>.
<p>
Otherwise,
emit a delim token
with its value set to U+002F SOLIDUS (/).
Remain in this state.
<dt>U+003A COLON (:)
<dd>
Emit a colon token.
Remain in this state.
<dt>U+003B SEMICOLON (;)
<dd>
Emit a semicolon token.
Remain in this state.
<dt>U+003C LESS-THAN SIGN (<)
<dd>
If the <i title="next input character">next 3 input characters</i> are
U+0021 EXCLAMATION MARK
U+002D HYPHEN-MINUS
U+002D HYPHEN-MINUS
(!--),
consume them
and emit a cdo token.
Remain in this state.
<p>
Otherwise,
emit a delim token
with its value set to U+003C LESS-THAN SIGN (<).
Remain in this state.
<dt>U+0040 COMMERCIAL AT (@)
<dd>
Switch to the <i>at-keyword state</i>.
<dt>U+005B LEFT SQUARE BRACKET ([)
<dd>
Emit a [ token.
Remain in this state.
<dt>U+005C REVERSE SOLIDUS (\)
<dd>
If the <i>next input character</i> is
a <i>newline</i>
or EOF,
this is a <i>parse error</i>.
Emit a delim token
with its value set to U+005C REVERSE SOLIDUS (\).
Remain in this state.
<p>
Otherwise, switch to the <i>ident state</i>.
Reconsume the <i>current input character</i>.
<dt>U+005D RIGHT SQUARE BRACKET (])
<dd>
Emit a ] token.
Remain in this state.
<dt>U+007B LEFT CURLY BRACKET ({)
<dd>
Emit a { token.
Remain in this state.
<dt>U+007D RIGHT CURLY BRACKET (})
<dd>
Emit a } token.
Remain in this state.
<dt><i>digit</i>
<dd>
Switch to the <i>number state</i>.
Reconsume the <i>current input character</i>.
<dt>U+0055 LATIN CAPITAL LETTER U (U)
<dt>U+0075 LATIN SMALL LETTER U (u)
<dd>
If the <i title="next input character">next 2 input character</i> are
U+002B PLUS SIGN (+)
followed by a <i>hex digit</i>,
consume the <i>next input character</i>.
<span class='note'>Note: don't consume both of them.</span>
Switch to the <i>unicode-range state</i>.
<p>
Otherwise,
if the <i title="next input character">next 3 input characters</i> are
U+0052 LATIN CAPITAL LETTER R (R) or U+0072 LATIN SMALL LETTER R (r)
followed by U+004C LATIN CAPITAL LETTER L (L) or U+006C LATIN SMALL LETTER L (l)
followed by U+0028 LEFT PARENTHESIS ((),
consume them.
Switch to the <i>url state</i>.
<p>Otherwise,
switch to the <i>ident state</i>.
Reconsume the <i>current input character</i>.
<dt><i>name-start character</i>
<dd>
Switch to the <i>ident state</i>.
Reconsume the <i>current input character</i>.
<dt>EOF
<dd>
Emit an end-of-file token.
<dt>anything else
<dd>
Emit a delim token
with its value set to the <i>current input character</i>.
Remain in this state.
</dl>
<h4>
<dfn>Double-quote-string state</dfn></h4>
<p>
If a string token has not yet been created since entering this state,
create a string token
with its value initially set to the empty string.
<p>
Consume the <i>next input character</i>.
<dl>
<dt>U+0022 QUOTATION MARK (")
<dd>
Emit the string token.
Return to the <i>data state</i>.
<dt>EOF
<dd>
This is a <i>parse error</i>.
Emit the string token.
Switch to the <i>data state</i>.
Reconsume the <i>current input character</i>.
<dt><i>newline</i>
<dd>
This is a <i>parse error</i>.
Emit a bad-string token.
Switch to the <i>data state</i>.
Reconsume the <i>current input character</i>.
<dt>U+005C REVERSE SOLIDUS (\)
<dd>
If the <i>next input character</i>
is an EOF,
this is a <i>parse error</i>.
Emit a bad-string token,
then switch to the <i>data state</i>.
<p>
Otherwise,
if the <i>next input character</i> is
a <i>newline</i>,
consume it.
Remain in this state.
<p>
Otherwise,
<i>consume an escaped character</i>.
Append the returned character to the string token's value.
Remain in this state.
<dt>anything else
<dd>
Append the <i>current input character</i>
to the string token's value.
Remain in this state.
</dl>
<h4>
<dfn>Single-quote-string state</dfn></h4>
<p>
If a string token has not yet been created since entering this state,
create a string token
with its value initially set to the empty string.
<p>
Consume the <i>next input character</i>.
<dl>
<dt>U+0027 APOSTROPHE (')
<dd>
Emit the string token.
Return to the <i>data state</i>.
<dt>EOF
<dd>
This is a <i>parse error</i>.
Emit the string token.
Switch to the <i>data state</i>.
Reconsume the <i>current input character</i>.
<dt><i>newline</i>
<dd>
This is a <i>parse error</i>.
Emit a bad-string token.
Switch to the <i>data state</i>.
Reconsume the <i>current input character</i>.
<dt>U+005C REVERSE SOLIDUS (\)
<dd>
If the <i>next input character</i>
is an EOF,
this is a <i>parse error</i>.
Emit a bad-string token,
then switch to the <i>data state</i>.
<p>
Otherwise,
if the <i>next input character</i> is
a <i>newline</i>,
consume it.
Remain in this state.
<p>
Otherwise,
<i>consume an escaped character</i>.
Append the returned character to the string token's value.
Remain in this state.
<dt>anything else