[indexes] Some CSS definitions incorrectly linking to the HTML spec

[SimonSapin]

In https://drafts.csswg.org/indexes/#properties, the line-height property for example links to its definition in HTML, where it is only defined by reference to CSS for the purpose of other parts of the HTML spec. This link should probably go to CSS2 instead.

html.spec.whatwg.org occurs 137 times in the HTML code of the index, so it is likely that some other definitions are also not going to the right place.

(https://lists.w3.org/Archives/Public/www-style/2018Jul/0035.html lead me to look at this.)

[tabatkins]

Note that that "line-height" link is actually in the Values section, because for some reason Shepherd's heuristics are tagging it as a value definition rather than a property. @plinss?

That said, 'line-height' doesn't show up in Properties at all, even tho it has definition from CSS2. Will look into this.

[tabatkins]

For reference, here's what Bikeshed currently knows:

> bikeshed refs --text=line-height
status:    snapshot
for:       []
level:     1
url:       https://www.w3.org/TR/CSS21/visudet.html#propdef-line-height
type:      property
export:    True
text:      line-height
shortname: css2
normative: True
spec:      css2
==========
status:    current
for:       []
level:     1
url:       https://drafts.csswg.org/css2/visudet.html#propdef-line-height
type:      property
export:    True
text:      line-height
shortname: css2
normative: True
spec:      css2
==========
status:    snapshot
for:       []
level:     1
url:       https://html.spec.whatwg.org/multipage/infrastructure.html#'line-height'
type:      value
export:    False
text:      line-height
shortname: html
normative: True
spec:      html

[tabatkins]

Ughhhhhh, I found the issue.

So, my code to deal with one spec obsoleting another is pretty simple - if spec A obsoletes spec B, and A shows up in the results, I automatically throw away any refs from B.

This works exactly as intended when you're, say, looking for a specific property - I want the 'color' definition from Colors, not from CSS2. It doesn't work so well when I'm asking for all properties, which I do when generating the index - at least one of the specs that obsoletes part of CSS2 shows up in the results, so I'm just throwing away all of the CSS2 refs automatically. Thus, anything defined only in CSS2 isn't showing up in the index.

[tabatkins]

Actually, the above isn't the issue. In fact, I'm not throwing away obsoleted refs at all when generating indexes (but if I did turn that on, it would make the problem much worse!).

Writing this down for my future self to have the information, as I can't work on this today:

• The actual code causing the current problem is the "latest only" filter when you're searching for refs - if the set of refs contains refs from multiple levels of a given spec, it throws away all refs from the lower levels. This is exactly what you want when you're searching for a single term; it's not at all what you want when you're trying to grab all terms.
• This is exacerbated by the fact that I've got some weird lingering references in the ref database. This is because my bikeshed update code correctly updates any new or changed files, but it never deletes any data files, so if some file was generated in the past but would no longer be generated, it just sticks around forever. In particular, at some point I was accidentally adding "heading" refs to the anchor database, and a few files were only filled with such references (in particular, anything of the form anchors-\d\d, like anchors-11, contained a bunch of references to headings for sections 1.1 of specs, and anchors-w3 contained a bunch of <h1> elements that started with "W3C Working Draft..." etc.
  • Relevant to this issue, CSS2.2 (in the database as shortname "css2", level "2") had some heading refs in the database, but no other refs. This caused the "latest version only" filter to trigger, throwing away all "css" level 1 references.

---

So I need to:

• Make the update code remove files that are no longer being generated.
• Make index generation work better, by finding all the terms I want to display with some filters turned off, then doing individual searches for each term with the filters turned on, so the end result shows the references you'd expect, exactly what you'd get if you just autolinked to the term.

Both of these fixes are relatively easy, and will fix the CSS2.1 issues, and probably quite a bit of other issues people haven't really noticed yet.

[tabatkins]

Okay, looks like the issue is cleared up - I regenerated Indexes and the 2017/2018 snapshots, and they look much better now. (For example, 2017, which tries to include Color 3 in its index, now actually includes values from Color 3.)