Meeting minutes
Minutes
<kaz> Day 3 (June 9)
McCool: typos
… (and then reviews the minutes)
… "Having a quite option" to be removed
… any other comments?
(none)
approved
TD Testing
McCool: would give some progress
PR 139
PR 139 - WIP: Update automated TD validation/testing
McCool: (goes through the changes)
… (specifically the update.sh script)
McCool: use GitHub feature to checkout another repo's content
… the current tools directory content is a bit old
current tools directory under the wot-testing repo
McCool: use "td.jsonld" as the suffix for TD
… (for the new update.sh)
… script for assertion tester there
… failed without input
… should show "do nothing" instead
… this (npm part) is the old way and need to be updated
… test_output area is empty and just have .keep as a placeholder
… removed "README.md" from the TDs area to avoid confusion
McCool: (shows the resources under Intel area)
proposed wot-testingevents/2021.06.Online/TD/TDs/Intel/ area
McCool: (then shows how the update.sh script works)
… (got some errors)
… still doesn't work well and need fixes
… need to figure out what's happening
Ege: issues with node packages?
McCool: need to look into the detail
… merge script might be broken
… any way that is the status
McCool: would like to see which assertion to be checked manually
… (shows intel-ocf.csv as an example)
McCool: let's create an issue to cover the problems
… have to update the term about what assertion means
… we're not really clear about the testing procedure yet
Ege: the speck might be also somewhat vague
McCool: right. the test effort would uncover possible issues
PR 135
PR 135 - test files for GH action validation - DO NOT MERGE
McCool: testing done
(closed)
Test plan
<McCool> https://
McCool: that is an old one
implementation report for ver 1.0
McCool: we need to generate an implementation report planning document?
Kaz: for the CR transition itself, we used the draft of the implementation report document itself for the planning purposes
McCool: what about our own planning?
… (looks into the plan.md page again)
McCool: this is kind of obsolete
Ege: we have some illegal input as well
McCool: that's good
McCool: we need to create an implementation plan
Kaz: yeah
… clarifying what to do is great
… also would be nice to define how to do
… e.g., how to apply which tool to what data
… then the result to be included in the report.html
McCool: yeah
… (creates an issue about testing plan)
Ege: some question about TM
McCool: ok
… we should put into the template
Question on TM
Ege: what if possible extension for a TM?
… our testing mechanism would allow that?
… could be multiple
… should think about something a bit advanced too
McCool: (shows Siemens' example)
MyLight-Extends-LwM2M.tm.jsonld
Sebastian: is the link to the previous Testfest still alive?
reference to TMs under 2021.03.Online
McCool: it's still alive
… note that Siemens is the only example
… we can also test manual assertions
… the implementation doesn't have to be a TM implementation
… could be an implementation for the resulted TD
… if I implement a TM and there is a property mentioning "ThingModel", that is an implementation for TM
… a command line level implementation would be useful
Ege: should we provide both TM and resulted TD?
McCool: all we need is a script that gives TDs
… TDs are aligned with some TM
Sebastian: wondering how it could be automated
… validation against a TM
… also validation based on JSON schema
Ege: property template and also another template
Ege: there is already some tool to be applicable but not necessarily optimized
McCool: would be nice to have a CLI tool like check-model for TD.tm.jsonld
<Ege> https://
McCool: output should be a CSV file
Kaz: how many assertions are related to TM?
… and then think about how to check them
Ege: 16 assertions or so
… some of them don't have any patterns
McCool: should check "type" and "extends" relation types in the links
… note that currently the edi(TD)or tool can do the check
… but it needs to be run manually
… using a GUI
… and tests only if the terms are used
Sebastian: there is a plan for further check
McCool: why don't we use the implementation for our testing purposes in general
… could be reused for the CLI checker as well
McCool: Siemens is encouraged to update the TD
… Oracle, etc., as well
Sebastian: Koster as well
Issue 141 has been created
Discovery testing
Farshid: put some information on slack
<FarshidT_> https://
McCool: great
… can deal with this
… list of assertions here
Farshid: bunch of sub tests
McCool: this is the test for LinkSmart?
Farshid: yeah
… there is a archived file to be download there
McCool: good start
Kaz: how to tell the relationship between the assertions?
… which is sub assertion of which?
McCool: each test to be assigned to some specific assertion for a feature within the spec
Farshid: note there is some negative tests as well
McCool: anyway this is a good start
… really appreciate
… next step is my cleaning up the script
… first draft report for tomorrow
… and then clarify the manual tests
Cristiano: any discussion about DNS-SD?
… implemented some tests
McCool: do you have any links?
McCool: let's look at this next time
Cristiano: ok
McCool: will update the manual tests for Intel as well
… please send your information too
… will incorporate it into the repo
[adjourned]