forked from w3c/csswg-drafts
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathOverview.html
More file actions
2442 lines (1827 loc) · 99.6 KB
/
Copy pathOverview.html
File metadata and controls
2442 lines (1827 loc) · 99.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang=en>
<head><meta content="text/html; charset=utf-8" http-equiv=Content-Type>
<title>CSS Syntax Module Level 3 (CSS3 Syntax)</title>
<link href="#contents" rel=contents>
<link href="#index" rel=index>
<link href="../default.css" rel=stylesheet type="text/css">
<link href="http://www.w3.org/StyleSheets/TR/W3C-ED.css" rel=stylesheet
type="text/css">
<style type="text/css">
/* delete this block when you've cleared out all the .replaceme elements */
.replaceme {
border: solid red;
padding: 0 0.5em;
margin: 0 0.1em;
background: yellow;
color: green;
}
.replaceme:before {
content: "FIXME(";
color: black;
}
.replaceme:after {
content: ")";
color: black;
}
</style>
<body>
<div class=head> <!--begin-logo-->
<p><a href="http://www.w3.org/"><img alt=W3C height=48
src="http://www.w3.org/Icons/w3c_home" width=72></a> <!--end-logo-->
<h1>CSS Syntax Module Level 3</h1>
<h2 class="no-num no-toc" id=longstatus-date>Editor's Draft 27 April 2012</h2>
<dl>
<dt>This version:
<dd> <a
href="http://dev.w3.org/csswg/css3-syntax/">http://dev.w3.org/csswg/css3-syntax/</a>
<!--
<dt>Latest version:
<dd>
<a href="http://www.w3.org/TR/css3-syntax/">http://www.w3.org/TR/css3-syntax/</a>
-->
<dt>Editor's draft:
<dd> <a
href="http://dev.w3.org/csswg/css3-syntax/">http://dev.w3.org/csswg/css3-syntax/</a>
<dt>Previous version:
<dd> <a
href="http://www.w3.org/TR/2003/WD-css3-syntax-20030813/">http://www.w3.org/TR/2003/WD-css3-syntax-20030813/</a>
<dt>Issue Tracking:
<dd> <a
href="https://www.w3.org/Bugs/Public/buglist.cgi?product=CSS&component=Syntax">W3C
Bugzilla</a>
<dt>Feedback:
<dd> <a
href="http://lists.w3.org/Archives/Public/www-style/">www-style@w3.org</a>
with subject line “<kbd>[css3-syntax] <var>… message topic
…</var></kbd>”
<dt>Editors:
<dd class="h-card vcard"> <a class="p-name fn u-url url"
href="http://www.xanthir.com/" rel=author>Tab Atkins Jr.</a> (<span
class="p-org org">Google, Inc.</span>), <span
class="u-email email">jackalmage@gmail.com</span>
</dl>
<!--begin-copyright-->
<p class=copyright><a
href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright"
rel=license>Copyright</a> © 2012 <a href="http://www.w3.org/"><abbr
title="World Wide Web Consortium">W3C</abbr></a><sup>®</sup> (<a
href="http://www.csail.mit.edu/"><abbr
title="Massachusetts Institute of Technology">MIT</abbr></a>, <a
href="http://www.ercim.eu/"><abbr
title="European Research Consortium for Informatics and Mathematics">ERCIM</abbr></a>,
<a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. W3C <a
href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>,
<a
href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a>
and <a
href="http://www.w3.org/Consortium/Legal/copyright-documents">document
use</a> rules apply.</p>
<!--end-copyright-->
<hr title="Separator for header">
</div>
<h2 class="no-num no-toc" id=abstract>Abstract</h2>
<p> <a href="http://www.w3.org/TR/CSS/">CSS</a> is a language for
describing the rendering of structured documents (such as HTML and XML) on
screen, on paper, in speech, etc. This module describes, in general terms,
the basic structure and syntax of CSS stylesheets. It defines, in detail,
the syntax and parsing of CSS - how to turn a stream of bytes into a
meaningful stylesheet.
<h2 class="no-num no-toc" id=status>Status of this document</h2>
<!--begin-status-->
<p>This is a public copy of the editors' draft. It is provided for
discussion only and may change at any moment. Its publication here does
not imply endorsement of its contents by W3C. Don't cite this document
other than as work in progress.
<p>The (<a
href="http://lists.w3.org/Archives/Public/www-style/">archived</a>) public
mailing list <a
href="mailto:www-style@w3.org?Subject=%5Bcss3-syntax%5D%20PUT%20SUBJECT%20HERE">
www-style@w3.org</a> (see <a
href="http://www.w3.org/Mail/Request">instructions</a>) is preferred for
discussion of this specification. When sending e-mail, please put the text
“css3-syntax” in the subject, preferably like this:
“[<!---->css3-syntax<!---->] <em>…summary of
comment…</em>”
<p>This document was produced by the <a href="/Style/CSS/members">CSS
Working Group</a> (part of the <a href="/Style/">Style Activity</a>).
<p>This document was produced by a group operating under the <a
href="/Consortium/Patent-Policy-20040205/">5 February 2004 W3C Patent
Policy</a>. W3C maintains a <a href="/2004/01/pp-impl/32061/status"
rel=disclosure>public list of any patent disclosures</a> made in
connection with the deliverables of the group; that page also includes
instructions for disclosing a patent. An individual who has actual
knowledge of a patent which the individual believes contains <a
href="/Consortium/Patent-Policy-20040205/#def-essential">Essential
Claim(s)</a> must disclose the information in accordance with <a
href="/Consortium/Patent-Policy-20040205/#sec-Disclosure">section 6 of the
W3C Patent Policy</a>.</p>
<!--end-status-->
<p>The following features are at risk: …
<h2 class="no-num no-toc" id=contents> Table of contents</h2>
<!--begin-toc-->
<ul class=toc>
<li><a href="#intro"><span class=secno>1. </span> Introduction</a>
<ul class=toc>
<li><a href="#placement"><span class=secno>1.1. </span> Module
interactions</a>
</ul>
<li><a href="#syntax-description"><span class=secno>2. </span> Description
of CSS's Syntax</a>
<li><a href="#parsing"><span class=secno>3. </span> Parsing</a>
<ul class=toc>
<li><a href="#overview-of-the-parsing-model"><span class=secno>3.1.
</span> Overview of the Parsing Model</a>
<li><a href="#the-input-byte-stream"><span class=secno>3.2. </span> The
input byte stream</a>
<ul class=toc>
<li><a href="#determining-the-character-encoding"><span
class=secno>3.2.1. </span> Determining the character encoding</a>
<li><a href="#preprocessing-the-input-stream"><span class=secno>3.2.2.
</span> Preprocessing the input stream</a>
</ul>
<li><a href="#parse-state"><span class=secno>3.3. </span> Parse
State</a>
<li><a href="#tokenization"><span class=secno>3.4. </span>
Tokenization</a>
<ul class=toc>
<li><a href="#tokenizer-flags"><span class=secno>3.4.1. </span>
Tokenizer Flags</a>
<li><a href="#definitions"><span class=secno>3.4.2. </span>
Definitions</a>
<li><a href="#data-state"><span class=secno>3.4.3. </span> Data
state</a>
<li><a href="#double-quote-string-state"><span class=secno>3.4.4.
</span> Double-quote-string state</a>
<li><a href="#single-quote-string-state"><span class=secno>3.4.5.
</span> Single-quote-string state</a>
<li><a href="#hash-state"><span class=secno>3.4.6. </span> Hash
state</a>
<li><a href="#hash-rest-state"><span class=secno>3.4.7. </span>
Hash-rest state</a>
<li><a href="#comment-state"><span class=secno>3.4.8. </span> Comment
state</a>
<li><a href="#at-keyword-state"><span class=secno>3.4.9. </span>
At-keyword state</a>
<li><a href="#at-keyword-rest-state"><span class=secno>3.4.10. </span>
At-keyword-rest state</a>
<li><a href="#identifier-state"><span class=secno>3.4.11. </span>
Identifier state</a>
<li><a href="#identifier-rest-state"><span class=secno>3.4.12. </span>
Identifier-rest state</a>
<li><a href="#transform-function-whitespace-state"><span
class=secno>3.4.13. </span> Transform-function-whitespace state</a>
<li><a href="#number-state"><span class=secno>3.4.14. </span> Number
state</a>
<li><a href="#number-rest-state"><span class=secno>3.4.15. </span>
Number-rest state</a>
<li><a href="#number-fraction-state"><span class=secno>3.4.16. </span>
Number-fraction state</a>
<li><a href="#dimension-state"><span class=secno>3.4.17. </span>
Dimension state</a>
<li><a href="#sci-notation-state"><span class=secno>3.4.18. </span>
Sci-notation state</a>
<li><a href="#url-state"><span class=secno>3.4.19. </span> URL
state</a>
<li><a href="#url-double-quote-state"><span class=secno>3.4.20.
</span> URL-double-quote state</a>
<li><a href="#url-single-quote-state"><span class=secno>3.4.21.
</span> URL-single-quote state</a>
<li><a href="#url-end-state"><span class=secno>3.4.22. </span> URL-end
state</a>
<li><a href="#url-unquoted-state"><span class=secno>3.4.23. </span>
URL-unquoted state</a>
<li><a href="#unicode-range-state"><span class=secno>3.4.24. </span>
Unicode-range state</a>
<li><a href="#consume-an-escaped-character"><span class=secno>3.4.25.
</span> Consume an escaped character</a>
<li><a href="#set-the-unicode-range-tokens-range"><span
class=secno>3.4.26. </span> Set the unicode-range token's range</a>
</ul>
<li><a href="#tree-construction"><span class=secno>3.5. </span> Tree
Construction</a>
<ul class=toc>
<li><a href="#creating-and-inserting-rules"><span class=secno>3.5.1.
</span> Creating and Inserting Rules</a>
<li><a href="#terms"><span class=secno>3.5.2. </span> Terms</a>
<li><a href="#top-level-insertion-mode-"><span class=secno>3.5.3.
</span> Top-level insertion mode </a>
</ul>
</ul>
<li><a href="#conformance"><span class=secno>4. </span> Conformance</a>
<ul class=toc>
<li><a href="#conventions"><span class=secno>4.1. </span> Document
conventions</a>
<li><a href="#conformance-classes"><span class=secno>4.2. </span>
Conformance classes</a>
<li><a href="#partial"><span class=secno>4.3. </span> Partial
implementations</a>
<li><a href="#experimental"><span class=secno>4.4. </span> Experimental
implementations</a>
<li><a href="#testing"><span class=secno>4.5. </span> Non-experimental
implementations</a>
<li><a href="#cr-exit-criteria"><span class=secno>4.6. </span> CR exit
criteria</a>
</ul>
<li class=no-num><a href="#acknowledgments"> Acknowledgments</a>
<li class=no-num><a href="#references"> References</a>
<ul class=toc>
<li class=no-num><a href="#normative-references"> Normative
references</a>
<li class=no-num><a href="#other-references"> Other references</a>
</ul>
<li class=no-num><a href="#index"> Index</a>
<li class=no-num><a href="#property-index"> Property index</a>
</ul>
<!--end-toc-->
<h2 id=intro><span class=secno>1. </span> Introduction</h2>
<p><em>This section is not normative.</em>
<p><var class=replaceme>Provide background, motivation, etc.</var>
<h3 id=placement><span class=secno>1.1. </span> Module interactions</h3>
<p> This module defines the syntax and parsing of CSS stylesheets. It
supersedes the lexical scanner and grammar defined in CSS 2.1.
<h2 id=syntax-description><span class=secno>2. </span> Description of CSS's
Syntax</h2>
<p><em>This section is not normative.</em>
<p> A CSS document is a series of <a href="#rule"><i>rules</i></a>, which
apply CSS properties to elements in the source document, and <a
href="#at-rules"><i>at-rules</i></a>, which define special processing
rules or values for the CSS document.
<p> A <dfn id=rule>rule</dfn> starts with a selector (defined by the <a
href="http://www.w3.org/TR/selectors/">Selectors specification</a>), then
has a {}-wrapped block containing a sequence of declarations. The selector
specifies which elements the declarations will apply to. Each declaration
has a property name, followed by a colon and the property value, and
finished with a semicolon.
<div class=example>
<p> A typical rule might look something like this:
<pre>
p > a {
color: blue;
text-decoration: underline;
}</pre>
<p> In the above rule, "<code>p > a</code>" is the selector, which, if the
source document is HTML, selects any <code><a></code> elements that
are children of a <code><p></code> element.
<p> "<code>color: blue;</code>" is a declaration specifying that, for the
elements that match the selector, their ‘<code
class=property>color</code>’ property should have the value
‘<code class=css>blue</code>’. Similiarly, their ‘<code
class=property>text-decoration</code>’ property should have the
value ‘<code class=css>underline</code>’.
</div>
<p> <dfn id=at-rules title=at-rule>At-rules</dfn> are all different, but
they have a basic structure in common. They start with an "@" character
followed by their name. Some <a href="#at-rules"><i>at-rules</i></a> are
simple statements, with their name followed by more CSS values to specify
their behavior, and finally ended by a semicolon. Others are blocks; they
can have CSS values following their name, but they end with a {}-wrapped
block, similar to a <a href="#rule"><i>rule</i></a>. Even the contents of
these blocks are specific to the given <a
href="#at-rules"><i>at-rule</i></a>: sometimes they contain a sequence of
declarations, like a <a href="#rule"><i>rule</i></a>; other times, they
may contain additional blocks, or at-rules, or other structures
altogether.
<div class=example>
<p> Here are several examples of <a href="#at-rules"><i>at-rules</i></a>
that illustrate the varied syntax they may contain.
<pre>@import "my-styles.css";</pre>
<p> The ‘<code class=css>@import</code>’ <a
href="#at-rules"><i>at-rule</i></a> is a simple statement. After its
name, it takes a single string or ‘<code
class=css>url()</code>’ function to indicate the stylesheet that it
should import.
<pre>
@page :left {
margin-left: 4cm;
margin-right: 3cm;
}</pre>
<p> The ‘<code class=css>@page</code>’ <a
href="#at-rules"><i>at-rule</i></a> consists of an optional page selector
(the ":left" pseudoclass), followed by a block of properties that apply
to the page when printed. In this way, it's very similar to a normal <a
href="#rule"><i>rule</i></a>, except that its properties don't apply to
any "element", but rather the page itself.
<pre>
@media print {
body { font-size: 10pt }
}</pre>
<p> The ‘<code class=css>@media</code>’ <a
href="#at-rules"><i>at-rule</i></a> begins with a media type and a list
of optional media queries. Its block contains entire rules, which are
only applied when the ‘<code class=css>@media</code>’s
conditions are fulfilled.
</div>
<p> Property names and <a href="#at-rules"><i>at-rule</i></a> names are
always <b>identifiers</b>, which have to start with a letter or a hyphen
followed by a letter, and then can contain letters, numbers, hyphens, or
underscores. You can include any character at all, even ones that CSS uses
in its syntax, by escaping it with a backslash (\) or by using a
hexadecimal escape.
<p> The syntax of selectors is defined in the <a
href="http://www.w3.org/TR/selectors/">Selectors spec</a>. Similarly, the
syntax of the wide variety of CSS values is defined in the <a
href="http://www.w3.org/TR/css3-values/">Values & Units spec</a>. The
special syntaxes of individual <a href="#at-rules"><i>at-rules</i></a> can
be found in the specs that define them.
<h2 id=parsing><span class=secno>3. </span> Parsing</h2>
<p> User agents must use the parsing rules described in this section to
generate the CSSOM trees from text/css resources. Together, these rules
define what is referred to as the CSS parser.
<p> This specification defines the parsing rules for CSS documents, whether
they are syntactically correct or not. Certain points in the parsing
algorithm are said to be parse errors. The error handling for parse errors
is well-defined: user agents must either act as described below when
encountering such problems, or must abort processing at the first error
that they encounter for which they do not wish to apply the rules
described below.
<p> Conformance checkers must report at least one parse error condition to
the user if one or more parse error conditions exist in the document and
must not report parse error conditions if none exist in the document.
Conformance checkers may report more than one parse error condition if
more than one parse error condition exists in the document. Conformance
checkers are not required to recover from parse errors.
<h3 id=overview-of-the-parsing-model><span class=secno>3.1. </span>
Overview of the Parsing Model</h3>
<p> The input to the CSS parsing process consists of a stream of Unicode
code points, which is passed through a tokenization stage followed by a
tree construction stage. The output is a CSSStyleSheet object.
<p class=note> Implementations that do not support scripting do not have to
actually create a CSSOM CSSStyleSheet object, but the CSSOM tree in such
cases is still used as the model for the rest of the specification.
<h3 id=the-input-byte-stream><span class=secno>3.2. </span> The input byte
stream</h3>
<p> The stream of Unicode code points that comprises the input to the
tokenization stage will be initially seen by the user agent as a stream of
bytes (typically coming over the network or from the local file system).
The bytes encode the actual characters according to a particular character
encoding, which the user agent must use to decode the bytes into
characters.
<p> The <i>encoding sniffing algorithm</i> defined below is used to
determine the character encoding.
<p> Given an encoding, the bytes in the input byte stream must be converted
to Unicode code points for the tokenizer's input stream, as described by
the rules for that encoding, except that the leading U+FEFF BYTE ORDER
MARK character, if any, must not be stripped by the encoding layer (it is
stripped by the rule below).
<p> Bytes or sequences of bytes in the original byte stream that could not
be converted to Unicode code points must be converted to U+FFFD
REPLACEMENT CHARACTERs. Specifically, if the encoding is UTF-8, the bytes
must be decoded with the error handling defined in this specification.
<p class=note> Bytes or sequences of bytes in the original byte stream that
did not conform to the encoding specification (e.g. invalid UTF-8 byte
sequences in a UTF-8 input byte stream) are errors that conformance
checkers are expected to report.
<h4 id=determining-the-character-encoding><span class=secno>3.2.1. </span>
Determining the character encoding</h4>
<div class=issue>
<p> General algorithm:
<ol>
<li> If the stylesheet is embedded, the charset is the charset of the
outer document.
<li> If the resource carried a Charset header, and the value of the
header is a valid charset, that's the charset of the stylesheet.
<li> If the document contains an @charset rule at the very top (specify
how to parse this in the presence of arbitrary charsets) and it
specifies a valid charset, that's the charset of the stylesheet.
<li> The charset is UTF-8.
</ol>
</div>
<h4 id=preprocessing-the-input-stream><span class=secno>3.2.2. </span>
Preprocessing the input stream</h4>
<p> The input stream consists of the characters pushed into it as the input
byte stream is decoded.
<p> One leading U+FEFF BYTE ORDER MARK character must be ignored if any are
present in the input stream.
<p class=note> The requirement to strip a U+FEFF BYTE ORDER MARK character
regardless of whether that character was used to determine the byte order
is a willful violation of Unicode, motivated by a desire to increase the
resilience of user agents in the face of naïve transcoders.
<p> U+000D CARRIAGE RETURN (CR) characters and U+000A LINE FEED (LF)
characters are treated specially. All CR characters must be converted to
LF characters, and any LF characters that immediately follow a CR
character must be ignored. Thus, newlines in CSS stylesheets are
represented by LF characters, and there are never any CR characters in the
input to the tokenization stage.
<p> The <dfn id=next-input-character>next input character</dfn> is the
first character in the input stream that has not yet been consumed or
explicitly ignored by the requirements in this section. Initially, the <a
href="#next-input-character"><i>next input character</i></a> is the first
character in the input. The <dfn id=current-input-character>current input
character</dfn> is the last character to have been consumed.
<p> The "EOF" character in the tables below is a conceptual character
representing the end of the input stream.
<h3 id=parse-state><span class=secno>3.3. </span> Parse State</h3>
<p> ...
<h3 id=tokenization><span class=secno>3.4. </span> Tokenization</h3>
<p> Implementations must act as if they used the following state machine to
tokenize HTML. The state machine must start in the <a
href="#data-state0"><i>data state</i></a>. Most states consume a single
character, which may have various side-effects, and either switches the
state machine to a new state to reconsume the same character, or switches
it to a new state to consume the next character, or stays in the same
state to consume the next character. Some states have more complicated
behavior and can consume several characters before switching to another
state.
<p> The output of the tokenization step is a series of zero or more of the
following tokens: identifier, function, at-keyword, hash, string,
bad-string, url, bad-url, delim, number, percentage, dimension,
unicode-range, whitespace, cdo, cdc, colon, semicolon, open-brace,
close-brace, open-paren, close-paren, open-bracket, close-bracket.
<p> Identifier, function, at-keyword, hash, string, and url tokens have a
value composed of zero or more characters. Delim tokens have a value
composed of a single character. Number, percentage, and dimension tokens
have a representation composed of 1 or more character, a numeric value,
and a type flag set to either "integer" or "number". The type flag
defaults to "integer" if not otherwise set. Dimension tokens additionally
have a unit composed of one or more characters. Unicode-range tokens have
a range of characters.
<p> The tokenizer state machine consists of the states defined in the
following subsections.
<h4 id=tokenizer-flags><span class=secno>3.4.1. </span> Tokenizer Flags</h4>
<p> The tokenizer can be run with any of several flags that alter its
behavior.
<dl>
<dt>the <dfn id=supports-scientific-notation>supports scientific
notation</dfn> flag
<dd> This flag is set if the tokenizer supports scientific notation in
numbers. When this is set, a sequence of characters like "12e3" will
tokenize as a number token with the value "12000". (Without the flag,
they would tokenize as a dimension token with value "12" and unit "e3".)
<p> Currently, this flag is only set when parsing SVG presentational
attributes.
<dt>the <dfn id=transform-function-whitespace>transform function
whitespace</dfn> flag
<dd> This flag is set when parsing SVG's <code>transform</code> attribute.
When this is set, whitespace is allowed between the name of a transform
function and its opening parenthesis.
</dl>
<h4 id=definitions><span class=secno>3.4.2. </span> Definitions</h4>
<p> This section defines several terms used during the tokenization phase.
<dl>
<dt><dfn id=digit>digit</dfn>
<dd> A character between U+0030 DIGIT ZERO (0) and U+0039 DIGIT NINE (9).
<dt><dfn id=hex-digit>hex digit</dfn>
<dd> A <a href="#digit"><i>digit</i></a>, or a character between U+0041
LATIN CAPITAL LETTER A (A) and U+0046 LATIN CAPITAL LETTER F (F), or a
character between U+0061 LATIN SMALL LETTER A (a) and U+0066 LATIN SMALL
LETTER F (f).
<dt><dfn id=uppercase-letter>uppercase letter</dfn>
<dd> A character between U+0041 LATIN CAPITAL LETTER A (A) and U+005A
LATIN CAPITAL LETTER Z (Z).
<dt><dfn id=lowercase-letter>lowercase letter</dfn>
<dd> A character between U+0061 LATIN SMALL LETTER A (a) and U+007A LATIN
SMALL LETTER Z (z).
<dt><dfn id=letter>letter</dfn>
<dd> An <a href="#uppercase-letter"><i>uppercase letter</i></a> or a <a
href="#lowercase-letter"><i>lowercase letter</i></a>.
<dt><dfn id=non-ascii-character>non-ASCII character</dfn>
<dd> A character with a codepoint equal to or greater than U+00A0 NO-BREAK
SPACE.
<dt><dfn id=name-start-character>name-start character</dfn>
<dd> A <a href="#letter"><i>letter</i></a>, a <a
href="#non-ascii-character"><i>non-ASCII character</i></a>, or U+005F LOW
LINE (_).
<dt><dfn id=name-character>name character</dfn>
<dd> A <a href="#name-start-character"><i>name-start character</i></a>, A
<a href="#digit"><i>digit</i></a>, or U+002D HYPHEN-MINUS (-).
<dt><dfn id=non-printable-character>non-printable character</dfn>
<dd> A character between U+0000 NULL and U+0008 BACKSPACE or a character
between U+000E SHIFT OUT and U+001F INFORMATION SEPARATOR ONE or a
character between U+007F DELETE and U+009F APPLICATION PROGRAM COMMAND.
<dt><dfn id=newline>newline</dfn>
<dd> U+000A LINE FEED or U+000C FORM FEED. <span class=note> Note that
U+000D CARRIAGE RETURN is not included in this definition, as it is
removed from the stream during preprocessing. </span>
<dt><dfn id=whitespace>whitespace</dfn>
<dd>A <a href="#newline"><i>newline</i></a>, U+0009 CHARACTER TABULATION,
or U+0020 SPACE.
<dt><dfn id=maximum-allowed-codepoint>maximum allowed codepoint</dfn>
<dd>The greatest codepoint defined by Unicode. This is currently U+10FFFF.
</dl>
<h4 id=data-state><span class=secno>3.4.3. </span> <dfn id=data-state0>Data
state</dfn></h4>
<p> Consume the <a href="#next-input-character"><i>next input
character</i></a>.
<dl>
<dt><a href="#whitespace"><i>whitespace</i></a>
<dd> Consume as much <a href="#whitespace"><i>whitespace</i></a> as
possible. Emit a whitespace token. Remain in this state.
<dt>U+0022 QUOTATION MARK (")
<dd> Switch to the <a
href="#double-quote-string-state0"><i>double-quote-string state</i></a>.
<dt>U+0023 NUMBER SIGN (#)
<dd> Switch to the <a href="#hash-state0"><i>hash state</i></a>.
<dt>U+0027 APOSTROPHE (')
<dd> Switch to the <a
href="#single-quote-string-state0"><i>single-quote-string state</i></a>.
<dt>U+0028 LEFT PARENTHESIS (()
<dd> Emit an open-paren token. Remain in this state.
<dt>U+0029 RIGHT PARENTHESIS ())
<dd> Emit a close-paren token. Remain in this state.
<dt>U+002B PLUS SIGN (+)
<dd> If the <a href="#next-input-character"><i>next input
character</i></a> is a <a href="#digit"><i>digit</i></a>, or the <a
href="#next-input-character"><i title="next input character">next 2 input
characters</i></a> are U+002E FULL STOP (.) followed by a <a
href="#digit"><i>digit</i></a>, switch to the <a
href="#number-state0"><i>number state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<p> Otherwise, emit a delim token with its value set to U+002B PLUS SIGN
(+). Remain in this state.
<dt>U+002D HYPHEN-MINUS (-)
<dd> If the <a href="#next-input-character"><i
title="next input character">next 2 input characters</i></a> are U+002D
HYPHEN-MINUS U+003E GREATER-THAN SIGN (->), consume them, emit a CDC
token, and remain in this state.
<p> Otherwise, if the <a href="#next-input-character"><i>next input
character</i></a> is a <a href="#digit"><i>digit</i></a>, or the <a
href="#next-input-character"><i title="next input character">next 2
input characters</i></a> are U+002E FULL STOP (.) followed by a <a
href="#digit"><i>digit</i></a>, switch to the <a
href="#number-state0"><i>number state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<p> Otherwise, if the <a href="#next-input-character"><i>next input
character</i></a> is a <a href="#name-start-character"><i>name-start
character</i></a>, switch to the <a
href="#identifier-state0"><i>identifier state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<p> Otherwise, emit a delim token with its value set to U+002D
HYPHEN-MINUS (-). Remain in this state.
<dt>U+002E FULL STOP (.)
<dd> If the <a href="#next-input-character"><i>next input
character</i></a> is a digit, switch to the <a
href="#number-state0"><i>number state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<p> Otherwise, emit a delim token with its value set to U+002E FULL STOP
(.). Remain in this state.
<dt>U+002F SOLIDUS (/)
<dd> If the <a href="#next-input-character"><i>next input
character</i></a> is U+002A ASTERISK (*), consume it and switch to the <a
href="#comment-state0"><i>comment state</i></a>.
<p> Otherwise, emit a delim token with its value set to U+002F SOLIDUS
(/). Remain in this state.
<dt>U+003A COLON (:)
<dd> Emit a colon token. Remain in this state.
<dt>U+003B SEMICOLON (;)
<dd> Emit a semicolon token. Remain in this state.
<dt>U+003C LESS-THAN SIGN (<)
<dd> If the <a href="#next-input-character"><i
title="next input character">next 3 input characters</i></a> are U+0021
EXCLAMATION MARK U+002D HYPHEN-MINUS U+002D HYPHEN-MINUS (!--), consume
them and emit a cdo token. Remain in this state.
<p> Otherwise, emit a delim token with its value set to U+003C LESS-THAN
SIGN (<). Remain in this state.
<dt>U+0040 COMMERCIAL AT (@)
<dd> Switch to the <a href="#at-keyword-state0"><i>at-keyword
state</i></a>.
<dt>U+005B LEFT SQUARE BRACKET ([)
<dd> Emit an open-bracket token. Remain in this state.
<dt>U+005C REVERSE SOLIDUS (\)
<dd> If the <a href="#next-input-character"><i>next input
character</i></a> is a <a href="#newline"><i>newline</i></a> or EOF, this
is a <i>parse error</i>. Emit a delim token with its value set to U+005C
REVERSE SOLIDUS (\). Remain in this state.
<p> Otherwise, switch to the <a href="#identifier-state0"><i>identifier
state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<dt>U+005D RIGHT SQUARE BRACKET (])
<dd> Emit a close-bracket token. Remain in this state.
<dt>U+007B LEFT CURLY BRACKET ({)
<dd> Emit an open-brace token. Remain in this state.
<dt>U+007D RIGHT CURLY BRACKET (})
<dd> Emit a close-brace token. Remain in this state.
<dt><a href="#digit"><i>digit</i></a>
<dd> Switch to the <a href="#number-state0"><i>number state</i></a>.
Reconsume the <a href="#current-input-character"><i>current input
character</i></a>.
<dt>U+0055 LATIN CAPITAL LETTER U (U)
<dt>U+0075 LATIN SMALL LETTER U (u)
<dd> If the <a href="#next-input-character"><i
title="next input character">next 2 input character</i></a> are U+002B
PLUS SIGN (+) followed by a <a href="#hex-digit"><i>hex digit</i></a>,
consume the <a href="#next-input-character"><i>next input
character</i></a>. <span class=note>Note: don't consume both of
them.</span> Switch to the <a
href="#unicode-range-state0"><i>unicode-range state</i></a>.
<p> Otherwise, if the <a href="#next-input-character"><i
title="next input character">next 3 input characters</i></a> are U+0052
LATIN CAPITAL LETTER R (R) or U+0072 LATIN SMALL LETTER R (r) followed
by U+004C LATIN CAPITAL LETTER L (L) or U+006C LATIN SMALL LETTER L (l)
followed by U+0028 LEFT PARENTHESIS ((), consume them. Switch to the <a
href="#url-state0"><i>url state</i></a>.
<p>Otherwise, switch to the <a href="#identifier-state0"><i>identifier
state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<dt><a href="#name-start-character"><i>name-start character</i></a>
<dd> Switch to the <a href="#identifier-state0"><i>identifier
state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<dt>EOF
<dd> Emit an end-of-file token.
<dt>anything else
<dd> Emit a delim token with its value set to the <a
href="#current-input-character"><i>current input character</i></a>.
Remain in this state.
</dl>
<h4 id=double-quote-string-state><span class=secno>3.4.4. </span> <dfn
id=double-quote-string-state0>Double-quote-string state</dfn></h4>
<p> If a string token has not yet been created since entering this state,
create a string token with its value initially set to the empty string.
<p> Consume the <a href="#next-input-character"><i>next input
character</i></a>.
<dl>
<dt>U+0022 QUOTATION MARK (")
<dd> Emit the string token. Return to the <a href="#data-state0"><i>data
state</i></a>.
<dt>EOF
<dd> This is a <i>parse error</i>. Emit the string token. Switch to the <a
href="#data-state0"><i>data state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<dt><a href="#newline"><i>newline</i></a>
<dd> This is a <i>parse error</i>. Emit a bad-string token. Switch to the
<a href="#data-state0"><i>data state</i></a>.
<dt>U+005C REVERSE SOLIDUS (\)
<dd> If the <a href="#next-input-character"><i>next input
character</i></a> is an EOF, this is a <i>parse error</i>. Append a
U+005C REVERSE SOLIDUS (\) to the string token's value and emit the
string token, then switch to the <a href="#data-state0"><i>data
state</i></a>.
<p> Otherwise, if the <a href="#next-input-character"><i>next input
character</i></a> is a <a href="#newline"><i>newline</i></a>, consume
it. Remain in this state.
<p> Otherwise, <a href="#consume-an-escaped-character0"><i>consume an
escaped character</i></a>. Append the returned character to the string
token's value. Remain in this state.
<dt>anything else
<dd> Append the <a href="#current-input-character"><i>current input
character</i></a> to the string token's value. Remain in this state.
</dl>
<h4 id=single-quote-string-state><span class=secno>3.4.5. </span> <dfn
id=single-quote-string-state0>Single-quote-string state</dfn></h4>
<p> If a string token has not yet been created since entering this state,
create a string token with its value initially set to the empty string.
<p> Consume the <a href="#next-input-character"><i>next input
character</i></a>.
<dl>
<dt>U+0027 APOSTROPHE (')
<dd> Emit the string token. Return to the <a href="#data-state0"><i>data
state</i></a>.
<dt>EOF
<dd> This is a <i>parse error</i>. Emit the string token. Switch to the <a
href="#data-state0"><i>data state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<dt><a href="#newline"><i>newline</i></a>
<dd> This is a <i>parse error</i>. Emit a bad-string token. Switch to the
<a href="#data-state0"><i>data state</i></a>.
<dt>U+005C REVERSE SOLIDUS (\)
<dd> If the <a href="#next-input-character"><i>next input
character</i></a> is an EOF, this is a <i>parse error</i>. Append a
U+005C REVERSE SOLIDUS (\) to the string token's value and emit the
string token, then switch to the <a href="#data-state0"><i>data
state</i></a>.
<p> Otherwise, if the <a href="#next-input-character"><i>next input
character</i></a> is a <a href="#newline"><i>newline</i></a>, consume
it. Remain in this state.
<p> Otherwise, <a href="#consume-an-escaped-character0"><i>consume an
escaped character</i></a>. Append the returned character to the string
token's value. Remain in this state.
<dt>anything else
<dd> Append the <a href="#current-input-character"><i>current input
character</i></a> to the string token's value. Remain in this state.
</dl>
<h4 id=hash-state><span class=secno>3.4.6. </span> <dfn id=hash-state0>Hash
state</dfn></h4>
<p> Consume the <a href="#next-input-character"><i>next input
character</i></a>.
<dl>
<dt><a href="#name-character"><i>name character</i></a>
<dd> Create a hash token with its value set to the <a
href="#current-input-character"><i>current input character</i></a>.
Switch to the <a href="#hash-rest-state0"><i>hash-rest state</i></a>.
<dt>U+005C REVERSE SOLIDUS (\)
<dd> If the <a href="#next-input-character"><i>next input
character</i></a> is a <a href="#newline"><i>newline</i></a> or EOF, emit
a delim token with its value set to U+0023 NUMBER SIGN (#). Switch to the
<a href="#data-state0"><i>data state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<p> Otherwise, <a href="#consume-an-escaped-character0"><i>consume an
escaped character</i></a>. Create a hash token with its value set to the
returned character. Switch to the <a
href="#hash-rest-state0"><i>hash-rest state</i></a>.
<dt>anything else
<dd> Emit a delim token with its value set to U+0023 NUMBER SIGN (#).
Switch to the <a href="#data-state0"><i>data state</i></a>. Reconsume the
<a href="#current-input-character"><i>current input character</i></a>.
</dl>
<h4 id=hash-rest-state><span class=secno>3.4.7. </span> <dfn
id=hash-rest-state0>Hash-rest state</dfn></h4>
<p> Consume the <a href="#next-input-character"><i>next input
character</i></a>.
<dl>
<dt><a href="#name-character"><i>name character</i></a>
<dd> Append the <a href="#current-input-character"><i>current input
character</i></a> to the hash token's value. Remain in this state.
<dt>U+005C REVERSE SOLIDUS (\)
<dd> If the <a href="#next-input-character"><i>next input
character</i></a> is a <a href="#newline"><i>newline</i></a> or EOF, emit
the hash token. Switch to the <a href="#data-state0"><i>data
state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.
<p> Otherwise, <a href="#consume-an-escaped-character0"><i>consume an
escaped character</i></a>. Append the returned character to the hash
token's value. Remain in this state.
<dt>anything else
<dd> Emit the hash token. Switch to the <a href="#data-state0"><i>data
state</i></a>. Reconsume the <a
href="#current-input-character"><i>current input character</i></a>.