W3C

- DRAFT -

WoT TestFest

27 Feb 2019

Attendees

Present
Kaz_Ashimura, Michael_McCool, Ege_Korkan, Kunihiko_Toumura, Michael_Lagally, Taki_Kamiya, Toru_Kawaguchi, Tomoaki_Mizushima
Regrets
Matthias
Chair
McCool
Scribe
ege

Contents


<kaz> scribenick: ege

McCool: review where we are
... (shows implementation report)
... changes to the TD repo added new assertions
... most of them have to do with the context
... some of them can be tested with JSON Schema but some other ones like URL should be dereferencable, we cannot test with JSON Schema
... there are duplicates
... we need to create context at form and security level
... I think we have to assume to keep these context assertions

Kaz: this is great. thank you!
... on the other hand, please remember that for CR transition what is required is clarifying all the assertions based on the spec
... fullfilling all the assertions with concrete results is not required
... getting concrete results earlier is of course nice, though

Ege: can we leave all of them not tested for CR

McCool: we can test all of them manually
... at the worst case, we can ,ake them "features at risk"

Kaz: note that having 20 features as "features at risk" would not be good

McCool: these are all related to one issue, context
... some are, like scopes, are very easy to fix
... panasonic added some examples
... so if some other companies do that too, it will be fine
... ege do you know why this td-data-schema-objects is not tested

Ege: I don't know, I will write an issue for it

McCool: I should mark some as resolved
... this is related to some subitems, td-event-names
... there is also multi languages missing
... we also need uriVariables

Ege: urivariables in events are complicated

McCool: each time I see 0s it looks weird
... td-forms is not tested?
... i think we miss a test here
... ege you should go through this list here

Ege: yes I can do that

<Zakim> kaz, you wanted to ask about (1) whether this report.html is generated based on the CSV files manually generated (or using the assertion tester) and (2) if this result includes all the results from Princeton TestFest

Kaz: I just want to make sure
... are these manual or automatic

McCool: both

<McCool> https://github.com/w3c/wot/tree/master/testfest/2019-03-online/templates

McCool: (shows the directory)
... panasonic had some updates as well

Kaz: so you merged automatic and manual

McCool: yes
... (explains how the merging works)

Kaz: I wasn't so sure whether these were equal

McCool: they don't overlap

Kaz: it would be nice to explain the fact that the inputs directory includes both the manually tested CSVs and the automatically tested CSVs, and the covered areas of assertions are different between them (manually tested CSV and automatically tested CSV)

Ege: in the readme maybe?

McCool: documentation is there if you look at the update.sh script but it would be clearer to split the "inputs" area into 2 pieces, one for manually tested CSVs and another for automatically tested CSVs. let me think about that.

Ege: the links are not valid anymore in the descriptions
... we have /inputs instead of /TDs anymore

Lagally: something has changed

<McCool> https://github.com/w3c/wot-thing-description/tree/master/testing/inputs/implementations

McCool: the readme is not that important but the html descriptions are
... I will edit them
... I have highlight for all the assertions, gray by default and yellow if it is at risk

Lagally: why is that name assertion yellow

Ege: this must because of the name being similar

McCool: (explains the bug)

Lagally: there is something wrong with description as well?

McCool: it seems so
... this happens because of the render
... and I can't change it easily
... maybe it is because the data is not provided but still marked as risk

Lagally: can we go through the spec and do a sanity check?

McCool: there is the content-type as well
... we have two contentType at the output data
... but only one here
... somehow handling multiple assertions is not good
... highlighting is not being generated right
... I can go over them manually and change the ids
... it is kind of tricky

Lagally: it is important to know whether some specs will go out of the spec

Summary of Action Items

Summary of Resolutions

[End of minutes]

Minutes manually created (not a transcript), formatted by David Booth's scribe.perl version 1.154 (CVS log)
$Date: 2019/03/02 23:59:32 $