[css-syntax] Tokenizing hex colors following spec is ambiguous

[tbjgolden]

I have been attempting to write a CSS tokenizer following the steps listed here:

https://drafts.csswg.org/css-syntax-3/#consume-token

However, it seems that there's no way (that I can see) to tokenize a hex color according to the steps listed.

When reaching U+0023 NUMBER SIGN (#), it contains instructions to create a hash token, but the instructions do not include a way to tokenize a hex color into a series of tokens that could be reasonably interpreted as a color.

example 1: #1e2125 <delim-token #><number-token 1*10^2125> (which ends up being "rounded to Infinity")
example 2: #ee2125 <hash-token #ee2125>
example 3: #1a2125 <delim-token #><number-token 1><ident-token a2125>

I could not find anything in document that covers this.

---

link to my test case: https://github.com/tbjgolden/media-query-parser/blob/main/src/parse/lexicalAnalysis.test.ts#L183-L245

[tabatkins]

The # shouldn't parse into a delim token if it's a hex color. You'll hit the "U+0023 NUMBER SIGN (#)" clause (the third one in the list), see that it starts with an identifier code point, and then parse it according to the steps listed there, notably consuming the rest of the hash token via consume an identifier.

From your examples, I suspect the problem you might be running into is assuming that the part following the # has to be an identifier in and of itself; that is not actually what's required by the algorithm. An "identifier code point" is any character valid within an identifier, which includes digits; the "consume an identifier" algo also doesn't confirm that the stream forms a valid identifier, and specifically calls out that this is something the algo caller needs to do themselves if it's important, otherwise it just keeps pulling characters as long as they're ident characters. Putting these together, it'll gladly parse 1e2125 as an identifier and attach the results to the <hash-token>, same as your second example.