WoT Testfest - Day 4

10 June 2021


Andrea_Cimmino, Cristiano_Aguzzi, Daniel_Peintner, Ege_Korkan, Fady_Salama, Farshid_Tavakolizadeh, Kaz_Ashimura, Kunihiko_Toumura, Michael_McCool, Sebastian_Kaebisch, Tomoaki_Mizushima

Meeting minutes


<kaz> Day 3 (June 9)

McCool: typos
… (and then reviews the minutes)
… "Having a quite option" to be removed
… any other comments?



TD Testing

McCool: would give some progress

PR 139

PR 139 - WIP: Update automated TD validation/testing

McCool: (goes through the changes)
… (specifically the update.sh script)

update.sh script

McCool: use GitHub feature to checkout another repo's content
… the current tools directory content is a bit old

current tools directory under the wot-testing repo

McCool: use "td.jsonld" as the suffix for TD
… (for the new update.sh)
… script for assertion tester there
… failed without input
… should show "do nothing" instead
… this (npm part) is the old way and need to be updated
… test_output area is empty and just have .keep as a placeholder
… removed "README.md" from the TDs area to avoid confusion

proposed TDs area

McCool: (shows the resources under Intel area)

proposed wot-testingevents/2021.06.Online/TD/TDs/Intel/ area

McCool: (then shows how the update.sh script works)
… (got some errors)
… still doesn't work well and need fixes
… need to figure out what's happening

Ege: issues with node packages?

McCool: need to look into the detail
… merge script might be broken
… any way that is the status

McCool: would like to see which assertion to be checked manually
… (shows intel-ocf.csv as an example)


McCool: let's create an issue to cover the problems
… have to update the term about what assertion means
… we're not really clear about the testing procedure yet

Ege: the speck might be also somewhat vague

McCool: right. the test effort would uncover possible issues

PR 135

PR 135 - test files for GH action validation - DO NOT MERGE

McCool: testing done


Test plan

<McCool> https://github.com/w3c/wot-testing/blob/main/data/testing/plan.md

Test plan document

McCool: that is an old one

1st CR for TD ver. 1.0

implementation report for ver 1.0

McCool: we need to generate an implementation report planning document?

Kaz: for the CR transition itself, we used the draft of the implementation report document itself for the planning purposes

McCool: what about our own planning?
… (looks into the plan.md page again)


McCool: this is kind of obsolete

Ege: we have some illegal input as well

McCool: that's good

McCool: we need to create an implementation plan

Kaz: yeah
… clarifying what to do is great
… also would be nice to define how to do
… e.g., how to apply which tool to what data
… then the result to be included in the report.html

McCool: yeah
… (creates an issue about testing plan)

Ege: some question about TM

McCool: ok
… we should put into the template

issue 140 - Create Testing Plan

Question on TM

Ege: what if possible extension for a TM?
… our testing mechanism would allow that?
… could be multiple
… should think about something a bit advanced too

McCool: (shows Siemens' example)


Sebastian: is the link to the previous Testfest still alive?

reference to TMs under 2021.03.Online

McCool: it's still alive
… note that Siemens is the only example
… we can also test manual assertions
… the implementation doesn't have to be a TM implementation
… could be an implementation for the resulted TD
… if I implement a TM and there is a property mentioning "ThingModel", that is an implementation for TM
… a command line level implementation would be useful

Ege: should we provide both TM and resulted TD?

McCool: all we need is a script that gives TDs
… TDs are aligned with some TM

Sebastian: wondering how it could be automated
… validation against a TM
… also validation based on JSON schema

Ege: property template and also another template

Ege: there is already some tool to be applicable but not necessarily optimized

McCool: would be nice to have a CLI tool like check-model for TD.tm.jsonld

<Ege> https://www.liquid-technologies.com/online-json-to-schema-converter

McCool: output should be a CSV file

Kaz: how many assertions are related to TM?
… and then think about how to check them

Ege: 16 assertions or so
… some of them don't have any patterns

McCool: should check "type" and "extends" relation types in the links
… note that currently the edi(TD)or tool can do the check
… but it needs to be run manually
… using a GUI
… and tests only if the terms are used

Sebastian: there is a plan for further check

McCool: why don't we use the implementation for our testing purposes in general
… could be reused for the CLI checker as well

McCool: Siemens is encouraged to update the TD
… Oracle, etc., as well

Sebastian: Koster as well

Issue 141 has been created

Discovery testing

Farshid: put some information on slack


<FarshidT_> https://github.com/farshidtz/wot-discovery-testing/blob/3f2cd327e616978307a61c4d535f3f6de782e1a4/report.csv

McCool: great
… can deal with this
… list of assertions here

Farshid: bunch of sub tests

McCool: this is the test for LinkSmart?

Farshid: yeah
… there is a archived file to be download there

McCool: good start

Kaz: how to tell the relationship between the assertions?
… which is sub assertion of which?

McCool: each test to be assigned to some specific assertion for a feature within the spec

Farshid: note there is some negative tests as well

McCool: anyway this is a good start
… really appreciate
… next step is my cleaning up the script
… first draft report for tomorrow
… and then clarify the manual tests

Cristiano: any discussion about DNS-SD?
… implemented some tests

McCool: do you have any links?

DNS-SD testing

McCool: let's look at this next time

Cristiano: ok

McCool: will update the manual tests for Intel as well
… please send your information too
… will incorporate it into the repo


Minutes manually created (not a transcript), formatted by scribe.perl version 136 (Thu May 27 13:50:24 2021 UTC).