13:04:04 RRSAgent has joined #wot 13:04:04 logging to https://www.w3.org/2021/06/10-wot-irc 13:04:15 meeting: WoT Testfest - Day 4 13:06:15 chair: McCool 13:06:44 present+ Kaz_Ashimura, Michael_McCool, Andrea_Cimmino, Daniel_Peintner, Kunihiko_Toumura, Tomoaki_Mizushima 13:07:57 scribenick: kaz 13:08:03 topic: Minutes 13:08:27 mm: typos 13:09:14 ... (and then reviews the minutes) 13:09:49 present+ Cristiano_Aguzzi 13:10:20 ... "Having a quite option" to be removed 13:10:44 ... any other comments? 13:10:46 (none) 13:10:48 approved 13:11:20 Agenda: https://www.w3.org/WoT/IG/wiki/PlugFest_WebConf#Agenda_10.06.2021 13:11:50 topic: TD Testing 13:11:58 mm: would give some progress 13:12:42 subtopic: PR 139 13:12:53 -> https://github.com/w3c/wot-testing/pull/139 PR 139 - WIP: Update automated TD validation/testing 13:13:12 mm: (goes through the changes) 13:14:03 ... (specifically the update.sh script) 13:14:11 -> https://github.com/mmccool/wot-testing/blob/update-td-test-script/events/2021.06.Online/TD/update.sh update.sh script 13:14:30 mm: use GitHub feature to checkout another repo's content 13:15:10 ... the current tools directory content is a bit old 13:15:20 -> https://github.com/w3c/wot-testing/tree/main/tools current tools directory under the wot-testing repo 13:15:34 mm: use "td.jsonld" as the suffix for TD 13:15:51 ... (for the new update.sh) 13:16:07 ... script for assertion tester there 13:16:18 ... failed without input 13:16:33 ... should show "do nothing" instead 13:17:00 ... this (npm part) is the old way and need to be updated 13:17:22 ... test_output area is empty and just have .keep as a placeholder 13:18:06 ... removed "README.md" from the TDs area to avoid confusion 13:18:45 -> https://github.com/mmccool/wot-testing/tree/update-td-test-script/events/2021.06.Online/TD/TDs proposed TDs area 13:19:11 mm: (shows the resources under Intel area) 13:19:32 -> https://github.com/mmccool/wot-testing/tree/update-td-test-script/events/2021.06.Online/TD/TDs/Intel proposed wot-testingevents/2021.06.Online/TD/TDs/Intel/ area 13:20:04 mm: (then shows how the update.sh script works) 13:20:11 ... (got some errors) 13:20:22 ... still doesn't work well and need fixes 13:20:44 ... need to figure out what's happening 13:20:50 present+ Ege_Korkan 13:20:55 zakim, who is on the call? 13:20:55 Present: Kaz_Ashimura, Michael_McCool, Andrea_Cimmino, Daniel_Peintner, Kunihiko_Toumura, Tomoaki_Mizushima, Cristiano_Aguzzi, Ege_Korkan 13:21:19 ek: issues with node packages? 13:21:26 mm: need to look into the detail 13:21:41 ... merge script might be broken 13:21:49 ... any way that is the status 13:22:01 present+ Fady_Salama 13:22:25 mm: would like to see which assertion to be checked manually 13:22:49 ... (shows intel-ocf.csv as an example) 13:22:54 -> https://github.com/mmccool/wot-testing/blob/update-td-test-script/events/2021.06.Online/TD/TDs/Intel/intel-ocf.csv intel-ocf.csv 13:23:05 present+ Sebastian_Kaebisch 13:23:14 mm: let's create an issue to cover the problems 13:23:32 ... have to update the term about what assertion means 13:23:45 ... we're not really clear about the testing procedure yet 13:24:15 ek: the speck might be also somewhat vague 13:24:30 mm: right. the test effort would uncover possible issues 13:25:41 subtopic: PR 135 13:25:54 -> https://github.com/w3c/wot-testing/pull/135 PR 135 - test files for GH action validation - DO NOT MERGE 13:26:05 mm: testing done 13:26:06 (closed) 13:27:16 dape has joined #wot 13:27:19 ktoumura_ has joined #wot 13:27:20 McCool has joined #wot 13:27:23 topic: Test plan 13:27:23 https://github.com/w3c/wot-testing/blob/main/data/testing/plan.md 13:27:28 -> https://github.com/w3c/wot-testing/blob/main/data/testing/plan.md Test plan document 13:27:32 mm: that is an old one 13:27:37 Mizushima has joined #wot 13:27:53 Fady_Salama has joined #wot 13:30:53 -> https://www.w3.org/TR/2019/CR-wot-thing-description-20190516/ 1st CR for TD ver. 1.0 13:31:24 -> https://w3c.github.io/wot-thing-description/testing/report.html implementatio report for ver 1.0 13:34:35 mm: we need to generate an implementation report planning document? 13:35:07 kaz: for the CR transition itself, we used the draft of the implementation report document itself for the planning purposes 13:35:08 Ege has joined #wot 13:35:54 mm: what about our own planning? 13:36:03 ... (looks into the plan.md page again) 13:36:08 -> https://github.com/w3c/wot-testing/blob/main/data/testing/plan.md plan.md 13:36:15 mm: this is kind of obsolete 13:36:35 q+ 13:37:10 ek: we have some illegal input as well 13:37:14 mm: that's good 13:37:28 q+ 13:37:31 ack e 13:37:44 mm: we need to create an implementation plan 13:38:29 kaz: yeah 13:38:37 ... clarifying what to do is great 13:38:46 ... also would be nice to define how to do 13:38:57 ... e.g., how to apply which tool to what data 13:39:08 ... then the result to be included in the report.html 13:39:11 mm: yeah 13:39:27 ... (creates an issue about testing plan) 13:39:35 ack k 13:39:38 q+ Ege 13:39:40 ack e 13:39:51 ek: some question about TM 13:39:56 mm: ok 13:41:05 ... we should put into the template 13:41:29 -> https://github.com/w3c/wot-testing/issues/140 issue 140 - Create Testing Plan 13:41:34 topic: Question on TM 13:41:55 ek: what if possible extension for a TM 13:42:00 s/TM/TM?/ 13:42:13 ... our testing mechanism would allow that? 13:42:27 ... could be multiple 13:42:51 ... should think about something a bit advanced too 13:43:54 mm: (shows Simens' example) 13:44:10 -> https://github.com/w3c/wot-testing/blob/main/events/2021.06.Online/TD/TMs/Siemens/MyLight-Extends-LwM2M.tm.jsonld MyLight-Extends-LwM2M.tm.jsonld 13:44:30 sk: is the link to the previous Testfest still alive? 13:44:58 -> https://raw.githubusercontent.com/w3c/wot-testing/main/events/2021.03.Online/TMs/Siemens/LwM2M-Light-Control-TM.tm.jsonld reference to TMs under 2021.03.Online 13:45:02 mm: it's still alive 13:45:21 ... note that Siemens is the only example 13:45:47 ... we can also test manual assertions 13:46:11 ... the implementation doesn't have to be a TM implementation 13:46:22 ... could be an implementation for the resulted TD 13:46:50 ... if I implement a TM and there is a property mentioning "model", that is an implementation for TM 13:47:05 s/"model"/"ThingModel"/ 13:47:23 ... a command line level implementation would be useful 13:47:51 ek: should we provide both TM and resulted TD? 13:48:15 mm: all we need is a script that gives TDs 13:48:25 ... TDs are aligned with some TM 13:48:33 q? 13:48:48 sk: wondering how it could be automated 13:48:59 ... validation against a TM 13:49:17 ... also validation based on JSON schema 13:49:30 ek: property template and also another template 13:49:41 q+ 13:50:23 ek: there is already some tool to be applicable but not necessarily optimized 13:50:52 mm: would be nice to have a CLI tool like check-model for TD.tm.jsonld 13:51:02 https://www.liquid-technologies.com/online-json-to-schema-converter 13:51:34 mm: output should be a CSV file 13:51:40 q? 13:52:04 kaz: how many assertions are related to TM? 13:52:15 ... and then think about how to check them 13:52:26 ek: 16 assertions or so 13:52:47 ... some of them don't have any patterns 13:53:59 mm: should check "type" and "extends" relation types in the links 13:54:25 ... note that currently the edi(TD)or tool can do the check 13:54:34 ... but it needs to be run manually 13:54:40 ... using a GUI 13:54:48 ... and tests only if the terms are used 13:55:31 sk: there is a plan for further check 13:55:54 mm: why don't we use the implementation for our testing purposes in general 13:56:20 ... could be reused for the CLI checker as well 13:56:40 present+ Farshid_Tavakolizadeh 13:56:44 rrsagent, make log public 13:56:47 rrsagent, draft minutes 13:56:47 I have made the request to generate https://www.w3.org/2021/06/10-wot-minutes.html kaz 13:57:01 mm: Siemens is encouraged to update the TD 13:57:09 ... Oracle, etc., as well 13:57:22 sk: Koster as well 13:57:29 topic: Discovery testing 13:57:42 ft: put some information on slack 13:58:12 FarshidT_ has joined #wot 13:58:29 -> https://github.com/farshidtz/wot-discovery-testing/blob/3f2cd327e616978307a61c4d535f3f6de782e1a4/report.csv report.csv 13:58:32 https://github.com/farshidtz/wot-discovery-testing/blob/3f2cd327e616978307a61c4d535f3f6de782e1a4/report.csv 13:59:00 mm: great 13:59:16 ... can deal with this 13:59:22 ... list of assertions here 13:59:27 ft: bunch of sub tests 13:59:54 q+ 14:00:53 mm: this is the test for LinkSmart? 14:00:56 ft: yeah 14:01:17 ... there is a archived file to be download there 14:01:29 mm: good start 14:03:26 kaz: how to tell the relationship between the assertions? 14:03:37 ... which is sub assertion of which? 14:07:07 mm: each test to be assigned to some specific assertion for a feature within the spec 14:07:20 ft: note there is some negative tests as well 14:07:47 mm: anyway this is a good start 14:07:57 ... really appreciate 14:08:09 ... next step is my cleaning up the script 14:08:22 ... first draft report for tomorrow 14:08:33 ... and then clarify the manual tests 14:09:09 ca: any discussion about DNSd? 14:09:20 ... implemented some tests 14:09:26 mm: do you have any links? 14:09:54 s/DNSd/DNSD/ 14:10:13 -> https://github.com/farshidtz/wot-discovery-testing/tree/main/dnssd 14:10:55 s/dnssd/dnssd DNS-SD testing/ 14:11:02 s/DNSD/DNS-SD/ 14:11:21 mm: let's look at this next time 14:11:23 ca: ok 14:13:18 mm: will update the manual tests for Intel as well 14:15:10 ... please send your information too 14:15:25 ... will incorporate it into the repo 14:15:30 [adjourned] 14:15:41 rrsagent, draft minutes 14:15:41 I have made the request to generate https://www.w3.org/2021/06/10-wot-minutes.html kaz 16:02:00 Zakim has left #wot 17:03:35 dsr has joined #wot 17:34:53 dsr has joined #wot 17:52:44 dsr has joined #wot 18:12:12 dsr has joined #wot 18:31:07 dsr has joined #wot 18:50:24 dsr has joined #wot 19:06:55 dsr has joined #wot 19:23:24 dsr has joined #wot 19:52:30 dsr has joined #wot 20:08:44 dsr has joined #wot 20:27:02 dsr has joined #wot 21:02:04 dsr has joined #wot 21:56:09 dsr has joined #wot 22:51:58 kaz has joined #wot 22:57:14 dsr has joined #wot 23:57:59 dsr has joined #wot