Skip to content

Commit 312d710

Browse files
committed
Preserve comments in the token stream. :/
1 parent 25b3391 commit 312d710

2 files changed

Lines changed: 15 additions & 6 deletions

File tree

css3-syntax/Overview.html

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -607,7 +607,8 @@ <h3 id=tokenization><span class=secno>3.4. </span> Tokenization</h3>
607607
<p> The output of the tokenization step is a series of zero or more of the
608608
following tokens: identifier, function, at-keyword, hash, string,
609609
bad-string, url, bad-url, delim, number, percentage, dimension,
610-
unicode-range, whitespace, cdo, cdc, colon, semicolon, [, ], (, ), {, }.
610+
unicode-range, whitespace, comment, cdo, cdc, colon, semicolon, [, ], (,
611+
), {, }.
611612

612613
<p> Identifier, function, at-keyword, hash, string, and url tokens have a
613614
value composed of zero or more characters. Delim tokens have a value
@@ -1060,13 +1061,13 @@ <h4 id=comment-state><span class=secno>3.4.8. </span> <dfn
10601061
<dt>U+002A ASTERISK (*)
10611062

10621063
<dd> If the <a href="#next-input-character"><i>next input
1063-
character</i></a> is U+002F SOLIDUS (/), consume it, and switch to the <a
1064-
href="#data-state0"><i>data state</i></a>.
1064+
character</i></a> is U+002F SOLIDUS (/), consume it, emit a comment
1065+
token, and switch to the <a href="#data-state0"><i>data state</i></a>.
10651066
<p> Otherwise, do nothing and remain in this state.
10661067

10671068
<dt>EOF
10681069

1069-
<dd> This is a <i>parse error</i>. Switch to the <a
1070+
<dd> This is a <i>parse error</i>. Emit a comment token. Switch to the <a
10701071
href="#data-state0"><i>data state</i></a>. Reconsume the <a
10711072
href="#current-input-character"><i>current input character</i></a>.
10721073

@@ -2092,7 +2093,7 @@ <h3 id=tree-construction><span class=secno>3.6. </span> Tree Construction</h3>
20922093

20932094
<dd> The tokens that can appear in the tree are: identifier, hash, string,
20942095
url, delim, number, percentage, dimension, unicode-range, whitespace,
2095-
cdo, cdc, colon, semicolon, at-keyword, ], ), and }.
2096+
comment, cdo, cdc, colon, semicolon, at-keyword, ], ), and }.
20962097
<p class=note> This means that the following tokens emitted by the
20972098
tokenizer stage will <strong>not</strong> appear in the stylesheet
20982099
object: function, bad-string, bad-url, [, (, {. <span class=issue>I need
@@ -2682,7 +2683,9 @@ <h3 id=changes-from-css-2.1-core-grammar><span class=secno>3.7. </span>
26822683
browsers, please let me know as it's almost certainly unintentional.
26832684

26842685
<ol>
2685-
<li>
2686+
<li> No whitespace or comments are allowed between the DELIM(!) and
2687+
IDENT(important) tokens when processing an !important directive at the
2688+
end of a style rule.
26862689
</ol>
26872690
<!--
26882691

css3-syntax/Overview.src.html

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -355,6 +355,7 @@ <h3>
355355
dimension,
356356
unicode-range,
357357
whitespace,
358+
comment,
358359
cdo,
359360
cdc,
360361
colon,
@@ -885,6 +886,7 @@ <h4>
885886
If the <i>next input character</i> is
886887
U+002F SOLIDUS (/),
887888
consume it,
889+
emit a comment token,
888890
and switch to the <i>data state</i>.
889891

890892
<p>
@@ -895,6 +897,7 @@ <h4>
895897
<dt>EOF
896898
<dd>
897899
This is a <i>parse error</i>.
900+
Emit a comment token.
898901
Switch to the <i>data state</i>.
899902
Reconsume the <i>current input character</i>.
900903

@@ -2093,6 +2096,7 @@ <h3>
20932096
dimension,
20942097
unicode-range,
20952098
whitespace,
2099+
comment,
20962100
cdo,
20972101
cdc,
20982102
colon,
@@ -2728,6 +2732,8 @@ <h3>
27282732

27292733
<ol>
27302734
<li>
2735+
No whitespace or comments are allowed between the DELIM(!) and IDENT(important) tokens
2736+
when processing an !important directive at the end of a style rule.
27312737
</ol>
27322738

27332739

0 commit comments

Comments
 (0)