Re: What *is* a spec test? (musings on feature interop tests)

Le 2013-11-26 09:15, Mihai Balan a écrit :
> Hello,
> Ms2ger, Alan, Tobie, thanks a lot for chiming in. I'll reply point by
> point to your comments:
> 
> @ Ms2ger
> --------
> Using `<link rel=help>` solves only half the problem, namely 
> identifying
> all the spec (sections) a test is related to.
> 
> The other problem - and the one that I find more difficult to solve - 
> is
> where should these tests reside (i.e. in which test suite). In some
> instances this is obvious (e.g. Testing how a flex container fragments 
> is
> a flexbox and not a fragmentation test case, since the flex spec is the
> one that actually specifies this behaviour, or at least the one that
> introduces this new case). However, there are instances where the
> distinction is more difficult to make (e.g. Testing a particularly 
> tricky
> situation involving auto-sized regions and fragmentation might be a
> regions test as well as a fragmentation test, (also) because the two 
> specs
> reference each other).
> 
> Or does the build step for test suites includes all the tests that
> reference a particular spec (section)?

Yes. Peter Linss should confirm this. In your example, it should create 
a test in regions and in fragmentation.

"
A test can have multiple specification links
(...)
If the test is part of multiple test suites, link to the relevant 
sections of each spec.
"
coming from
http://testthewebforward.org/docs/test-templates.html#specification-links
and from
http://wiki.csswg.org/test/format#specification-links


> In which case, I guess the problem
> of the inclusion of a test in one suite or another is moot since tests
> will be automatically duplicated across test suite without the need to
> sync them "manually".
> 

Yes.

Although there is no CSS3-break or CSS3-fragmentation folder for source 
yet.

> @ Alan
> ------
> Having read the section on Interoperability in the Overview

Can you pinpoint the url where you read this?


> it seems there
> are two meanings for "interoperability" - which can lead to some 
> confusion.
> 
> The interoperability that the Overview mentions refers to the ability 
> of
> different CSS implementation to produce the same output given the same
> input. The interoperability I was talking about in my initial email 
> refers
> to something slightly different: the property of any 2+ CSS/HTML/JS
> modules/features that, when used together, produce a result that makes
> sense (it's not surprising to the end-user) and is non-equivocal and
> unique (e.g. two people reading the specs for the features involved 
> will
> not expect two different results).
> 
> To my knowledge, there's little talk of the latter, even though there 
> are
> numerous ways in which different CSS modules can interact which are not
> always neither obvious nor well specified.

Many non-basic tests involve more than 1 property in action, being 
involved in a test. In my mind, property interaction (or property 
inter-relation) and interoperability are 2 distinct words with 2 
distinct meanings.

> 
> This was the aspect that I was trying to get some feedback on: how to
> efficiently test for this kind of interoperability?
> 
> (Regarding the CSS3-Regions test suite, I'm one of the main 
> contributors
> to it and this was the very thing that prompted me to start this thread 
> :)
> )

"
This specification is related to other specifications as described in 
the references section. In addition, it is related to the following 
specifications:

     CSS Fragmentation Module Level 3 [CSS3-BREAK]. This module defines 
the rules for fragmenting content over multiple containers and applies 
to CSS regions in addition to applying to multi-column and paged media.
"
CSS3 Regions, §9. Relation to other specifications
http://www.w3.org/TR/css3-regions/#relation-to-other-specifications

Therefore, it seems normal to have tests that will involve other 
properties and other parts of other specs.



-- 
Web authors' contributions to CSS 2.1 test suite
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/web-authors-contributions-css21-testsuite.html
CSS 2.1 Test suite RC6, March 23rd 2011
http://test.csswg.org/suites/css2.1/20110323/html4/toc.html

Received on Tuesday, 26 November 2013 19:12:11 UTC